Online Unsupervised Learning of Visual Representations and Categories

1 INTRODUCTION . Human and artificial agents learn from large amounts of unlabeled data through a continuous stream of experience that has strong temporal correlations and hierarchical structure . The stream of experience is determined by the fact that individuals operate in distinct physical environments , such as home and office . Transitions between environments occur on a coarse time scale relative to the rate of encounters with objects in a given environment . The distribution over objects is strongly conditioned on the environment : frozen pizza and milk are in the supermarket , computer monitors and desks at the office . The distribution may also be nonstationary ; for example , a visitor may be infrequent , but when they are around , they ’ re encountered frequently . And finally , the distribution can be highly unbalanced : an individual may interact with their co-workers daily but the boss only occasionally . The goal of our research is to tackle the challenging problem of online unsupervised representation learning in the setting of environments with naturalistic structure . We desire a learning algorithm that will facilitate the categorization of objects encountered in the environment , with few or zero category labels . In representation learning , methods often evaluate their ability to classify from the representation using either supervised linear readout or unsupervised clustering over the full dataset , both of which are typically done in a separate post-hoc evaluation phase . An important aim of our work is to produce object-category predictions throughout training and evaluation , and to allow these predictions to guide subsequent categorization . Unsurprisingly , the structure of natural environments contrasts dramatically with the standard scenario typically assumed by many machine learning algorithms : mini-batches of independent and identically distributed ( iid ) samples from a well-curated dataset . In unsupervised visual representation learning , the most successful methods rely on iid samples . Contrastive-based objectives ( Chen et al. , 2020a ; He et al. , 2020 ) typically assume that each instance in the mini-batch forms its own instance class , throwing away the potential similarity between instances . Clustering-based learning frameworks ( Caron et al. , 2018 ; Asano et al. , 2020 ; Caron et al. , 2020 ) often assume that the set of cluster centroids remain relatively stable and that the clusters are balanced in size . Unfortunately , none of these assumptions necessarily hold true in a naturalistic online streaming setting . The performance of methods will suffer as a consequence . Contrastive approaches could eventually fail if examples of the same class are pushed apart ; clustering approaches will behave erratically with nonstationary and imbalanced class distributions , two key facets of the natural online non-iid learning we focus on here . To make progress on the challenge of unsupervised visual representation learning and categorization in a naturalistic setting , we propose the online unsupervised prototypical network , which performs learning of visual representations and object categories simultaneously in a single-stage process . Class prototypes are created via an online clustering procedure , and a contrastive loss ( van den Oord et al. , 2018 ) is used to encourage different views of the same image to be assigned to the same cluster . Notably , our online clustering procedure is more flexible relative to other clustering-based representation learning algorithms , such as DeepCluster ( Caron et al. , 2018 ) and SwAV ( Caron et al. , 2020 ) : our model performs learning and inference as an online Gaussian mixture model , where clusters can be created online with only a single new example , and cluster assignments do not have to be balanced , which permits an approximation to natural imbalanced distributions from uncurated raw data . We train and evaluate our algorithm on a recently proposed naturalistic learning dataset , RoamingRooms ( Ren et al. , 2021 ) , which uses imagery collected from a virtual agent walking through different rooms . Unlike the experiments in Ren et al . ( 2021 ) , our training is done without using any labeled data , and hence it is online and unsupervised . We compare to state-of-the-art unsupervised methods SimCLR ( Chen et al. , 2020a ) , SwAV ( Caron et al. , 2020 ) , and SimSiam ( Chen & He , 2021 ) . Because they rely on sampling a large batch from an offline dataset , their performance drops significantly using smaller non-iid episodes . In contrast , our method can directly learn with online streaming data from small episodes , without requiring an example buffer , and surprisingly , we even outperform these strong methods trained offline with large batches of iid data . We also used RoamingOmniglot ( Ren et al. , 2021 ) as a benchmark to investigate the effect of imbalanced classes and we find that our method is very robust to an imbalanced distribution of classes . For a version of ImageNet with non-iid structure , RoamingImageNet , we again outperform SimCLR and SwAV when using the same batch size . Finally , qualitative visualizations confirm that our online clustering procedure can automatically discover new concepts and categories by grouping a few similar instances together in the learned embedding space . 2 RELATED WORK . Self-supervised learning . Self-supervised learning methods discover rich and informative visual representations without class labels . Instance-based approaches aim to learn invariant representations of each image under different transformations ( van den Oord et al. , 2018 ; Misra & van der Maaten , 2020 ; Tian et al. , 2020 ; He et al. , 2020 ; Chen et al. , 2020a ; b ; Grill et al. , 2020 ; Chen & He , 2021 ) . They typically work well with iid data and large batch sizes , which contrasts with realistic learning scenarios . Our method is also related to clustering-based approaches , which obtain clusters on top of the learned embedding and use the cluster assignments to constrain the embedding network . To compute the cluster assignment , DeepCluster ( Caron et al. , 2018 ; Zhan et al. , 2020 ) and PCL ( Li et al. , 2021 ) use the k-means algorithm whereas SeLa ( Asano et al. , 2020 ) and SwAV ( Caron et al. , 2020 ) uses the Sinkhorn-Knopp algorithm ( Cuturi , 2013 ) . However , they typically assume a fixed number of clusters , and Sinkhorn-Knopp further assumes a balanced assignment as an explicit constraint . In contrast , our online clustering procedure is more flexible : it can create new clusters on-the-fly with only a single new example and does not assume balanced cluster assignments . Self-supervised pretraining or joint training has proven beneficial for online continual learning tasks ( Zhang et al. , 2020 ; Gallardo et al. , 2021 ; Cha et al. , 2021 ) . Representation learning from video . There has also been a surge of interest in leveraging video data to learn visual representations ( Wang & Gupta , 2015 ; Orhan et al. , 2020 ; Pathak et al. , 2017 ; Zhu et al. , 2020 ; Xiong et al. , 2021 ) . These approaches all sample video subsequences uniformly over the entire dataset , whereas our model directly learns from an online stream of data . Our model also does not have the assumption that inputs must be adjacent frames in the video . Online and incremental representation learning . Our work is also related to online and continual representation learning ( Rebuffi et al. , 2017 ; Castro et al. , 2018 ; Rao et al. , 2019 ; Jerfel et al. , 2019 ; Javed & White , 2019 ; Hayes et al. , 2020 ) . Continual mixture models ( Rao et al. , 2019 ; Jerfel et al. , 2019 ) designate a categorical latent variable that can be dynamically allocated for a new environment . Our model has a similar mixture latent variable setup but one major difference is that we operate on example-level rather than task-level . Streaming learning ( Hayes et al. , 2020 ; 2019 ) aims to perform representation learning online . Most work here except Rao et al . ( 2019 ) assumes a fully supervised Net setting . Our prototype memory also resembles a replay buffer ( Buzzega et al. , 2020 ; Kim et al. , 2020 ) , but we store the feature prototypes instead of the inputs . Latent variable modeling on sequential data . Our model also relates to a family of latent variable generative models for sequential data ( Johnson et al. , 2016 ; Krishnan et al. , 2015 ; He et al. , 2018 ; Denton & Fergus , 2018 ; Zhu et al. , 2020 ) . Like our model , these approaches aim to infer latent variables with temporal structure , but they use an input reconstruction criterion . Online mixture models . Our clustering module is related to the literature on online mixture models , e.g. , Bottou & Bengio ( 1995 ) ; Carpenter & Grossberg ( 1987 ) ; Anderson ( 1991 ) ; Hughes & Sudderth ( 2013 ) ; Pinto & Engel ( 2015 ) ; Song & Wang ( 2005 ) . Typically , these are designed for fast and incremental learning of clusters without having to recompute clustering over the entire dataset . Despite presenting a similar online clustering algorithm , our goal is to jointly learn both online clusters and input representations that facilitate future online clustering episodes . Few-shot learning . Our model can recognize new classes with only one or a few examples . Our prototype-based memory is also inspired by the Prototypical Network and its variants ( Snell et al. , 2017 ; Allen et al. , 2019 ; Ren et al. , 2021 ) . Few-shot methods can reduce or remove reliance on class labels using semi- and self-supervised learning ( Ren et al. , 2018 ; Huang et al. , 2019 ; Hsu et al. , 2019 ; Gidaris et al. , 2019 ; Antoniou & Storkey , 2019 ; Khodadadeh et al. , 2019 ; Medina et al. , 2020 ) . Classical few-shot learning , however , relies on episodes of equal number of training and test examples from a fixed number of new classes . Gidaris & Komodakis ( 2018 ) ; Triantafillou et al . ( 2020 ) ; Tao et al . ( 2020 ) ; Zhu et al . ( 2021 ) consider extending the standard episodes with incremental learning and varying number of examples and classes . Ren et al . ( 2021 ) proposed a new setup that incrementally accumulates new classes and re-visits old classes over a sequence of inputs . We evaluate our algorithm on a similar setup ; however , unlike that work , our proposed algorithm does not rely on any class labels . Human category learning . Our work is related to human learning settings and online clustering models from cognitive science ( Fisher et al. , 1991 ; Murphy , 2004 ; Carpenter & Grossberg , 1987 ; Anderson , 1991 ; Love et al. , 2004 ; Lake et al. , 2009 ) . In contrast to these human learning models , our model learns both representations and categories in an end-to-end fashion . 3 ONLINE UNSUPERVISED PROTOTYPICAL NETWORKS . In this section , we introduce our proposed model , online unsupervised prototypical networks ( OUPN ) . We study the online categorization setting , where the model receives an input xt at every time step t , and predicts both a categorical variable ŷt to indicate the object class and also a binary variable ût to indicate whether this is a known ( u = 0 ) or new ( u = 1 ) class . OUPN uses a network h to encode the input to obtain embedding zt = h ( xt ; θ ) , where θ represents the learnable parameters of the encoder network . z is then processed by a prototype memory which predicts ( ŷt , ût ) .

Summary: The authors propose a method for unsupervised learning of instance-based clustering which is more robust to data imbalance and non-iid distributions than existing approaches, such as SwAW. In particular, they propose an Expectation Maximization algorithm that operates in temporal episodes (supposed to correspond to an agent moving through an environment).  At every step an observation (image with an overplayed instance segmentation mask) is first encoded with a CNN and the resulting vector is assigned to one of the existing clusters based on the distance in the feature space, or a new cluster is created (E-step). This is formulated in a probabilistic framework. The cluster prototypes are then updated accordingly (M-step). The loss, computed after a sequence of such steps, consists of a standard contrastive loss (encourages assigning similar images to the same clusters) + entropy (encouraging confident assignments) + a term encouraging the creation of new clusters.   The method is mainly evaluated on RoamingRooms - a dataset with an agent moving through indoor environments with instance labels assigned to objects. The task is then to cluster different view points of the same object together. The main challenge is that the distribution of examples is non-iid, since in every episode mostly the same objects are seen. Additional evaluation on online variants of Omniglot and ImageNet is also provided.  When compared to offline and online contrastive learning algorithms the proposed approach demonstrates stronger performance especially with low batch size and non-iid distribution of examples.