Unsupervised Progressive Learning and the STAM Architecture
Abstract
We ﬁrst pose the Unsupervised Progressive
Learning (UPL) problem: an online representation learning problem in which the learner observes a non-stationary and unlabeled data stream, and identiﬁes a growing number of features that persist over time even though the data is not stored or replayed.
To solve the UPL problem we propose the Self-Taught Associative
Memory (STAM) architecture.
Layered hierarchies of STAM modules learn based on a combination of online clustering, novelty detection, forgetting outliers, and storing only prototypical features rather than speciﬁc examples.
We evaluate STAM representations using classiﬁcation and clustering tasks.
Even though there are no prior approaches that are directly applicable to the UPL problem, we evaluate the
STAM architecture in comparison to some unsupervised and self-supervised deep learning approaches adapted in the UPL context.
1. Introduction
The Continual Learning (CL) problem is predominantly addressed in the supervised context with the goal being to learn a sequence of tasks without “catastrophic forgetting” (Goodfellow et al., 2013; Parisi et al., 2019; van de
Ven & Tolias, 2019).
There are several CL variations but a common formulation is that the learner observes a set of examples { ( x i , t i , y i ) } , where x i is a feature vector,
t i is a task identiﬁer and y i is the target vector associated with ( x i , t i ) (Chaudhry et al., 2019a;b; LopezPaz & Ranzato, 2017).
Other CL variations replace task identiﬁers with task boundaries that are either given (Hsu et al., 2018) or inferred (Zeno et al., 2018).
Typically,
CL requires that the learner either stores and replays some previously seen examples (Aljundi et al., 2019a;b; Gepperth & Karaoguz, 2017; Hayes et al., 2019; Kemker et al., 2018; Rebufﬁ et al., 2017) or generates examples of earlier learned tasks (Kemker & Kanan, 2018; Liu et al., 2020;
Shin et al., 2017).
The Feature (or Representation) Learning (FL) problem, on the other hand, is unsupervised but mostly studied in the ofﬂine context : given a set of examples { x i } , the goal is to learn a feature vector (of a given, ﬁxed dimensionality) h
i = f
( x i ) that, ideally, makes it easier to identify the explanatory factors of variation behind the data (Bengio et al., 2013), leading to better performance in tasks such as classiﬁcation or clustering.
FL methods differ in the prior
P ( h ) and the loss function.
Autoencoders, for instance, aim to learn features of a lower dimensionality than the input that enable a sufﬁciently good reconstruction at the output (Bengio, 2014; Kingma & Welling, 2013; Tschannen et al., 2018;
Zhou et al., 2012).
A similar approach is self-supervised methods, which learn representations by optimizing an auxiliary task (Berthelot et al., 2019;
Doersch et al., 2015; Gidaris et al., 2018; Kuo et al., 2019; Oord et al., 2018; Sohn et al., 2020).
In this work, we focus on a new and pragmatic problem that adopts some elements of CL and FL but is also different than them – we refer to this problem as Unsupervised
Progressive Learning (UPL) .
UPL can be described as follows:
the data is observed as a non-IID stream (e.g., different portions of the stream may follow different distributions and there may be strong temporal correlations between successive examples), 2.
the features should be learned exclusively from unlabeled data, 3.
each example is “seen” only once and the unlabeled data are not stored for iterative processing, 4.
the number of learned features may need to increase over time, in response to new tasks and/or changes in the data distribution, 5. to avoid catastrophic forgetting, previously learned features need to persist over time, even when the corresponding data are no longer observed in the stream.
The UPL problem is encountered in important AI applications, such as a robot learning new visual features as it explores a time-varying environment.
Additionally, we argue that UPL is closer to how animals learn, at least in the case of perceptual learning (Goldstone, 1998).
We believe that in order to mimic that, ML methods should be able to learn in a streaming manner and in the absence of supervision.
Animals do not “save off” labeled examples to train in parallel with unlabeled data, they do not know how many “classes” exist in their environment, and they do not have to replay/dream periodically all their past experiences to avoid forgetting them.
To the extent of our knowledge, the UPL problem has not been addressed before.
The closest prior work is CURL (“Continual Unsupervised Representation Learning”) by
Rao et al.
(Rao et al., 2019).
CURL however does not impose the requirement that the data is presented to the learner as a stream that should be processed online, and so CURL requires iterative processing through gradient minimization methods (additional differences with CURL are discussed in Section 6).
To address the UPL problem, we describe an architecture referred to as STAM (“Self-Taught Associative Memory”).
STAM learns features through online clustering at a hierarchy of increasing receptive ﬁeld sizes.
Online clustering can be performed through a single pass over the data stream.
Further, despite its simplicity, clustering can generate representations that enable better classiﬁcation performance than more complex FL methods such as sparsecoding or some deep learning methods (Coates et al., 2011;
Coates & Ng, 2012).
STAM allows the number of clusters to increase over time, driven by a novelty detection mechanism.
Additionally, STAM includes a brain-inspired dual-memory hierarchy (short-term versus long-term) that enables the conservation of previously learned features (to avoid catastrophic forgetting) that have been seen multiple times at the data stream, while forgetting outliers.
2. STAM Architecture
In the following, we describe the STAM architecture as a sequence of its major components: a hierarchy of increasing receptive ﬁelds, online clustering (centroid learning), novelty detection, and a dual-memory hierarchy that stores prototypical features rather than speciﬁc examples.
The notation is summarized in SM-A in the extended verison of the paper found under the same title on arXiv.
I. Hierarchy of increasing receptive ﬁelds: An input vector x t ∈ R n (an image in all subsequent examples) is analyzed through a hierarchy of Λ layers.
Instead of neurons or hidden-layer units, each layer consists of STAM units – in its simplest form a STAM unit functions as an online clustering module.
Each STAM processes one ρ l × ρ l patch (subvector) of the input at that layer.
The patches are overlapping, with a small stride (set to one pixel in our experiments) to accomplish translation invariance (similar to
CNNs).
The patch dimension ρ l increases in higher layers – the idea is that the ﬁrst layer learns the smallest and most elementary features while the top layer learns the largest and most complex features.
II.
Centroid Learning: Every patch of each layer is clustered, in an online manner, to a set of centroids.
These time-varying centroids form the features that the STAM architecture gradually learns at that layer.
All STAM units of layer l share the same set of centroids C l ( t ) – again for translation invariance.
Given the m ’th input patch x l , m at layer l , the nearest centroid of C l selected for x l , m is
[Equation]
where d ( x l , m , c ) is the Euclidean distance between the patch x l , m and centroid c .
The selected centroid is updated based on a learning rate parameter α , as follows:
[Equation]
A higher α value makes the learning process faster but less predictable.
We do not use a decreasing value of α because the goal is to keep learning in a non-stationary environment rather than convergence to a stable centroid.
III.
Novelty detection: When an input patch x l , m at layer l is signiﬁcantly different than all centroids at that layer (i.e., its distance to the nearest centroid is a statistical outlier), a new centroid is created in C l based on x l , m .
We refer to this event as Novelty Detection (ND) .
This function is necessary so that the architecture can learn novel features when the data distribution changes.
To do so, we estimate in an online manner the distance distribution between input patches and their nearest centroid (separately for each layer).
The novelty detection threshold at layer l is denoted by ˆ D l and it is deﬁned as the 95-th percentile ( β = 0.95) of this distance distribution.
IV.
Dual-memory organization:
New centroids are stored temporarily in a Short-Term Memory (STM) of limited ca- pacity ∆ , separately for each layer.
Every time a centroid is selected as the nearest neighbor of an input patch, it is updated based on (2).
If an STM centroid c l , j is selected more than θ times, it is copied to the Long-Term Memory (LTM) for that layer.
We refer to this event as memory con- solidation The
LTM has (practically) unlimited capacity and the
learning rate is
much
smaller (in our experiments the LTM learning rate is set to zero This memory organization is inspired by the Complementary Learning Systems framework (Kumaran et al., 2016), where the STM role is played by the hippocampus and the LTM role by the cortex.
This dual-memory scheme is necessary to distinguish between infrequently seen patterns that can be forgotten (”outliers”), and new patterns that are frequently seen after they ﬁrst appear (”novelty”).
We initialize the pool of STM centroids at each layer using randomly sampled patches from the ﬁrst few images of the unlabeled stream.
When the STM pool of centroids at a layer is full, the introduction of a new centroid (created through novelty detection) causes the removal of an earlier centroid.
We use the Least-Recently Used (LRU) policy to remove atypical centroids that have not been recently selected by any input.
Figure 1 illustrates this dual-memory organization.
3. Classiﬁcation using STAM
Given a small amount of labeled data, STAM can be used in classiﬁcation tasks.
We emphasize that the labeled data is not used for representation learning – it is only used to associate previously learned features with a given set of classes.
I. Associating centroids with classes: Suppose we are given some labeled examples X L ( t ) from a set of classes
L ( t ) at time t .
We can use these labeled examples to associate existing LTM centroids at time t (learned strictly from unlabeled data) with the set of classes in L ( t ) .
Given a labeled example of class k , suppose that there is a patch x in that example for which the nearest centroid is c .
That patch contributes the following association between centroid c and class k :
[Equation]
where ¯ D l is a normalization constant (calculated as the average distance between input patches and centroids).
The class-association vector g c between centroid c and any class k is computed aggregating all such associations, across all labeled examples in X L :
[Equation]
where X L ( k ) refers to labeled examples belonging to class k .
Note that (cid:80)
k g c ( k ) =1.
II.
Class informative centroids: If a centroid is associated with only one class k
( g c ( k ) = 1 ), only labeled examples of that class select that centroid.
At the other extreme, if a centroid is equally likely to be selected by examples of any labeled class, ( g c ( k ) ≈ 1 / | L ( t ) | ), the selection of that centroid does not provide any signiﬁcant information for the class of the corresponding input.
We identify the cen- troids that are Class INformative (CIN) as those that are associated with at least one class signiﬁcantly more than expected by chance.
Speciﬁcally, a centroid c is CIN if max
[Equation]
where 1 | L ( t ) | is the chance term and γ is the signiﬁcance term.
III.
Classiﬁcation using a hierarchy of centroids: At test time, we are given an input x of class k ( x ) and infer its class as ˆ k ( x ) .
The classiﬁcation task is a “biased voting” process in which every patch of x , at any layer, votes for a single class as long as that patch selects a CIN centroid.
Speciﬁcally, if a patch x l , m of layer l selects a CIN centroid c , then that patch votes v l,m ( k ) =
max k ∈ L ( t ) g c ( k ) for the class k that has the highest association with c , and zero for all other classes.
If c is not a CIN centroid, the vote of that patch is v l,m ( k ) = 0 for all classes.
The vote of layer l for class k is the average vote across all patches in layer l (as illustrated in Figure 2):
[Equation]
where M l is the set of patches in layer l .
The ﬁnal inference for input x is the class with the highest cumulative vote across all layers:
[Equation]
4. Clustering using STAM
We can also use STAM representations in unsupervised tasks, such as ofﬂine clustering.
To do this, we ﬁrst deﬁne an embedding function that maps a given vector x into the space deﬁned by STAM LTM centroids.
In particular, the embedding is deﬁned as Φ( x ) : R n −→ R | C | , where the element j = 1 ... | C | of Φ( x ) is the normalized distance (Equation equation 3) between the LTM centroid c l and its closest patch in x .
The embedding vector represents how strongly each feature (LTM centroid) is present anywhere in the given input.
The embedding vectors of a given dataset are then clustered ofﬂine using k -means for a given value of k .
Any other clustering algorithm could be used instead.
5. Evaluation
To evaluate the STAM architecture in the UPL context, we consider a data stream in which small groups of classes appear in successive phases , referred to as Incremental UPL .
New classes are introduced two at a time in each phase, and they are only seen in that phase.
STAM must be able to both recognize new classes when they are ﬁrst seen in the stream, and to also remember all previously learned classes without catastrophic forgetting.
Another evaluation scenario is Uniform UPL , where all classes appear with equal probability throughout the stream.
The results for Uniform UPL are shown in SM-F in the extended verison of the paper found under the same title on arXiv.
For brevity, we include results for three datasets: MNIST (Lecun et al., 1998), EMNIST (balanced split withclasses)
(Cohen et al., 2017), and SVHN (Netzer et al., 2011) (we have also experimented with CIFAR).
For each dataset we utilize the standard training and test splits.
We preprocess the images by applying per-patch normalization (instead of image normalization), and color images are transformed to grayscale.
More information about the image preprocessing can be found in SM-G in the extended verison of the paper found under the same title on arXiv.
We create the training stream by randomly selecting, with equal probability, N p data examples from the classes seen during each phase.
N p is set to 8000 , 8000 , and 2000 for
MNIST, SVHN, and EMNIST respectively.
More information about the impact of the stream size can be found in
SM-D in the extended verison of the paper found under the same title on arXiv.
In the classiﬁcation task, we select a small portion of the training dataset as the labeled examples that are available only to the classiﬁer.
In each task, we average results over three different unlabeled data streams.
During testing, we select 100 random examples of each class from the test dataset.
This process is repeated ﬁve times for each training stream (i.e., a total of ﬁfteen results per experiment).
The following plots show mean ± standard deviation ranges.
We utilize one F72s V2 and one NC6s V2 virtual machine from Microsoft’s Azure cloud computing service to perform all experiments.
For all datasets, we use a 3-layer STAM hierarchy.
The hyperparameter values are tabulated in SM-A in the extended verison of the paper found under the same title on arXiv.
The robustness of the results with respect to these values is shown in SM-E in the extended verison of the paper found under the same title on arXiv.
Baseline Methods: We evaluate the STAM architecture comparing its performance to few basic unsupervised and self-supervised models that we have adapted in the UPL context.
We emphasize that there are no prior approaches that are directly applicable to the UPL problem, and so we cannot perform direct comparisons between STAM and “competing” models.
In follow-up work, we plan to modify existing methods that were designed to address different problems (such as CURL, GEM or iCARL) in the UPL context and compare them with STAM.
The baselines we consider here are:
(I) a Convolutional AutoEncoder (CAE) trained to minimize Euclidean reconstruction error, (II) a rotation-based self-supervised method which learns the auxiliary task of predicting image rotations based on RotNet (Gidaris et al., 2018), and (III) ofﬂine Principal Component Analysis (PCA) , just for reference purposes, as it is probably the simplest baseline.
A detailed description of these baseline models is presented in SM-B in the extended verison of the paper found under the same title on arXiv.
To satisfy the stream requirement of UPL, the number of training epochs for the CAE and RotNet models is set to one.
This is necessary so that each unlabeled example is processed only once.
Deep learning methods become weaker in this streaming scenario because they cannot train iteratively over several epochs on the same dataset.
For all baselines, the classiﬁcation task is performed using a K nearest-neighbor (KNN) classiﬁer – we have experimented with various values of K and other single-pass classiﬁers, and report only the best performing results here.
We have also compared the memory requirement of STAM (storing centroids at STM and LTM) with the memory requirement of the CAE and RotNet baselines (storing neural network weights).
The results of that comparison appear in SM-H in the extended verison of the paper found under the same title on arXiv.
For instance, STAM has 17% of
RotNet’s memory footprint.
Classiﬁcation Task:
We focus on an expanding classiﬁcation task , meaning that in each phase we need to classify all classes seen so far.
The results for the classiﬁcation task are given in Figure 3.
Note that we use only 10 labeled examples per class for MNIST and EMNIST, and 100 examples per class for SVHN.
As we introduce new classes in the training stream, the average accuracy per phase decreases for all methods in each dataset.
This is expected, as the task gets more difﬁcult after each phase.
We focus on which method performs best in each task, and which methods see a smaller decrease in accuracy per phase.
In the ﬁrst dataset (MNIST), we observe that STAM performs consistently better than RotNet and
CAE, and STAM is less vulnerable to catastrophic forgetting.
For SVHN, the trend is similar after the ﬁrst phase but the difference between STAM and RotNet is much smaller.
Finally, in EMNIST, we see a consistently higher accuracy with STAM compared to the deep learning baselines.
For additional analysis and discussion of these results, please also refer to SM-C in the extended verison of the paper found under the same title on arXiv.
Clustering Task:
Given that we have the same number of test vectors per class, we associate each cluster with the most-represented class in that cluster.
Any instances of another class in that cluster are counted as errors.
The number of clusters k is equal to twice the number of classes seen up to that phase in the unlabeled datastream.
The results of the clustering task are given in Figure 4.
For MNIST, STAM still performs consistently better than the two other models, and its accuracy stays almost constant going from 4 classes to 10 classes.
For SVHN, RotNet performs signiﬁcantly better.
Finally, for EMNIST, STAM outperforms the two deep learning methods without experiencing signiﬁcant loss of accuracy after the ﬁrst 10 phases (20 classes).
Ablation studies: Several STAM ablations are presented in Figure 5.
On the left, we remove the LTM capability and only use STM centroids for classiﬁcation.
During the ﬁrst two phases, there is little (if any) difference in classiﬁcation accuracy.
However, we see a clear dropoff during phases 3-5.
This suggests that, without the LTM mechanisms, features from classes that are no longer seen in the stream are forgotten over time, and STAM can only successfully classify classes that have been recently seen.
We also investigate the importance of having static LTM centroids rather than dynamic centroids (Fig.
5-middle).
Speciﬁcally, we replace the static LTM with a dynamic
LTM in which the centroids are adjusted with the same learning rate parameter α , as in STM.
The accuracy suffers drastically because the introduction of new classes “takes over” LTM centroids of previously learned classes, after the latter are removed from the stream.
Similar to the removal of LTM, we do not see the effects of “forgetting” until phases 3-5.
Note that the degradation due to a dynamic LTM is less severe than that from removing LTM completely.
Finally, we look at the effects of removing layers from the
STAM hierarchy (Fig. 5-right).
We see a small drop in accuracy after removing layer 3, and a large drop in accuracy after also removing layer 2.
The importance of having a deeper hierarchy would be more pronounced in datasets with higher-resolution images or videos, potentially show- ing multiple objects in the same frame.
In such cases, CIN centroids can appear at any layer, starting from the lowest to the highest.
Additional results: The reader can ﬁnd additional experimental results in the Supp-Material section that focus on the questions: how does the number of LTM centroids increase with time and what fraction of them are “Class Informative” (in classiﬁcation tasks), how does the accuracy of
STAM vary with the number of labeled examples per class, and how do the various hyperparameters of the STAM architecture affect classiﬁcation accuracy?
6. Related Work
The UPL problem has some similarities with several recent approaches in the machine learning literature but it is also different in important aspects we describe in this section.
Each paragraph highlights the most relevant prior work and explains how it is different from UPL.
I: Continual learning:
In addition to CL models cited in the introduction, other supervised CL methods include regularization-based approaches (Aljundi et al., 2018;
Golkar et al., 2019; Hayes & Kanan, 2019; Kirkpatrick et al., 2017; Yoon et al., 2018; Zenke et al., 2017), expanding architectures (Lomonaco & Maltoni, 2017; Maltoni &
Lomonaco, 2019; Rusu et al., 2016), and distillation-based methods (Lee et al., 2019; 2020; Li & Hoiem, 2017).
Their main difference with UPL and STAM is that they are designed for supervised learning, and it is not clear how to adapt them for non-stationary and unlabeled data streams.
II.
Ofﬂine unsupervised learning: Additional ofﬂine representation learning methods include clustering (Caron et al., 2018; Jiang et al., 2017a;
Xie et al., 2016; Yang et al., 2016), generative models (Eslami et al., 2016; Jiang et al., 2017b; Kosiorek et al., 2018; 2019), information theory (Hjelm et al., 2019; Ji et al., 2019), among others.
These methods require prior information about the number of classes present in a given dataset (to set the number of cluster centroids or class outputs) and iterative training (i.e. data replay), and therefore cannot be directly applied in the UPL setting.
III.
Semi-supervised learning (SSL): SSSL methods require labeled data during the representation learning stage and so they are not compatible with UPL (Kingma et al., 2014; Lee, 2013;
Miyato et al., 2018; Oliver et al., 2018; Rasmus et al., 2015; Springenberg, 2015; Tarvainen & Valpola, 2017).
IV.
Few-shot learning (FSL) and Meta-learning: These methods recognize object classes not seen in the training set with only a single (or handful) of labeled examples (Fei-Fei et al., 2006;
Finn et al., 2017; Ren et al., 2018;
Snell et al., 2017).
Similar to SSL, FSL methods require labeled data to learn representations and therefore are not applicable in the UPL context.
Centroid networks (Huang et al., 2019) do not require labeled examples at inference time but require labeled examples for training.
V. Multi-Task Learning (MTL): Any MTL method that involves separate heads for different tasks is not compatible with UPL because task boundaries are not known a priori in UPL (Ruder, 2017).
MTL methods that require pre-training on a large labeled dataset are also not applicable to UPL (Pan & Yang, 2010;
Yosinski et al., 2014).
VI.
Online and Progressive Learning:
Many earlier methods learn in an online manner, meaning that data is processed in ﬁxed batches and discarded afterwards.
This includes progressive learning (Venkatesan & Er, 2016) and streaming with limited supervision (Chiotellis et al., 2018;
Li et al., 2018; Loo & Marsono, 2015), both of which require labeled data in the training stream.
VII.
Continual Unsupervised Representation Learning (CURL): Similar to STAM,
CURL also focuses on the problem of continual unsupervised learning from nonstationary data with unknown task boundaries (Rao et al., 2019).
Its main difference with STAM however is that it is not a streaming method, and so it does not require that each example is seen only once.
The CURL model requires gradient-based optimization, going through the same data multiple times.
Another difference with STAM is that catastrophic forgetting in CURL is addressed through a generative model that also needs to be learned.
VIII.
Data dimensionality and clustering-based representation learning : As mentioned earlier in this section, clustering has been used successfully in the past for ofﬂine representation learning (e.g., (Coates et al., 2011; Coates &
Ng, 2012)).
Its effectiveness, however, gradually drops as the input dimensionality increases (Beyer et al., 1999; Hinneburg et al., 2000).
In the STAM architecture, we avoid this issue by clustering smaller subvectors (patches) of the input data.
If those subvectors are still of high dimensionality, another approach is to reduce the intrinsic dimensionality of the input data at each layer by reconstructing that input using representations (selected centroids) from the previous layer.
VIII.
Related work to other STAM components : STAM relies on online clustering.
This algorithm can be implemented with a rather simple recurrent neural network of excitatory and inhibitory spiking neurons, as shown recently (Pehlevan et al., 2017).
The novelty detection component of STAM is related to the problem of anomaly detection in streaming data (Dasgupta et al., 2018)
— and the simple algorithm currently in STAM can be replaced with more sophisticated methods (e.g., (Cui et al., 2016;
Yong et al., 2012)).
Finally, brain-inspired dual-memory systems have been proposed before for memory consolidation (e.g., (Kemker & Kanan, 2018; Parisi et al., 2018; Shin et al., 2017)).
7. Discussion
The STAM architecture aims to address the following desiderata that is often associated with Lifelong Learning (Parisi et al., 2019):
I. Online learning: STAMs update the learned features with every observed example.
There is no separate training stage for speciﬁc tasks, and inference can be performed in parallel with learning.
II.
Transfer learning: The features learned by the STAM architecture in earlier phases can be also encountered in the data of future tasks (forward transfer).
Additionally, new centroids committed to LTM can also be closer to data of earlier tasks (backward transfer).
III.
Resistance to catastrophic forgetting: The STMLTM memory hierarchy of the STAM architecture mitigates catastrophic forgetting by committing to ”permanent storage” (LTM) features that have been often seen in the data during any time period of the training period.
IV.
Expanding learning capacity: The unlimited capacity of LTM allows the system to gradually learn more features as it encounters new classes and tasks.
The relatively small size of STM, on the other hand, forces the system to forget features that have not been recalled frequently enough after their creation.
V. No direct access to previous experience: STAM only needs to store data centroids in a hierarchy of increasing receptive ﬁelds – there is no need to store previous exemplars or to learn a generative model that can produce such examples.