Feature Transformers: A Unified Representation Learning Framework for Lifelong Learning

1 INTRODUCTION . Deep learning algorithms have achieved tremendous success on various challenging tasks like object detection , language translation , medical image segmentation , etc . Lifelong learning - the ability to adapt , benefit and sustain performance post deployment with more data and feedback , is an important goal for artificial intelligence ( AI ) and extremely crucial for sustained utility of these algorithms in domains like healthcare ( learning from rare cases ) , self driving ( learning new object detectors ) , etc . While there is no single standard definition of lifelong learning , most of the research in this field can be classified into one of the following sub-categories : 1 . Incremental learning - encountering new data , with no change in distribution , for same task 2 . Domain adaptation - data from modified target distributions but for the same task 3 . New-task learning - data from tasks that were not presented before Majority of the successful techniques study these variants in isolation . However , a realistic lifelong learning scenario would involve a mixture of these manifestations over time . Another major impediment to a successful lifelong learner is data privacy . In domains like healthcare , it is near impossible to have data access beyond scope ( both time and geography ) which leads to catastophic forgetting ( McCloskey & Cohen , 1989 ) - the inability of machine learning models to retain past knowledge while learning from new data . In this paper , we provide a unified framework - feature transformers - for practical lifelong learning to handle data privacy and catastrophic forgetting . In summary , the major contributions of our work are as follows : • Define a realistic and a challenging lifelong learning scenario and formulate a unique and generic mathematical framework - feature transformer , to work in this setting successfully . • Ensure data privacy by storing only features from previous episodes while successfully combating catastrophic forgetting . • Principled way of utilizing additional neural-compute to tackle complex incremental tasks . • Demonstrate state-of-the-art results on common continuous learning benchmarks for new task learning like MNIST rotations , iCIFAR dataset and a challenging healthcare problem . • Demonstrate exemplary performance even under severe constraints on memory and com- pute , thus operating under the assumptions of practical lifelong learning . Following review of related work in section 2 , we describe the mathematical formulation of feature transformers and their practical realization in section 3 . Section 4 contains experiments and results , followed by discussion in section 5 . 2 RELATED WORK . A conventional deep learning classifier can be viewed as an automatic feature extractor followed by a classifier , trained jointly . For lifelong learning , the methods in literature can be broadly classified into : 1 ) learning with fixed feature representations and 2 ) incrementally evolving representations . Fixed Represenations : In this class of approaches , a representation for the data is learnt from the initial task and remains frozen for ensuing tasks , while only the classification layers are modified . The simplest baseline approaches include fine-tuning ( Oquab et al. , 2014 ) and Mensink et al . ( 2012 ) where the idea of using fixed representations was extended further by using the nearest class mean classifier . In spite of their simplicity , these algorithms do not perform well in practice due to the limiting constraint of a fixed representation throughout the incremental learning phase . Incrementally evolving representations : The last few years have witnessed renewed interest in developing methods that allow changing data representation with the addition of new tasks . Naive methods of retraining with only new data suffer from catastrophic forgetting ( McCloskey & Cohen , 1989 ) . To overcome this problem , most methods have attempted different manifestations of rehearsal ( Robins , 1995 ) - replaying data from the previous tasks . However , access to all the previous data is usually not feasible due to size , compute and privacy concerns , researchers have attempted to reproduce past through proxy information , known as pseudorehearsal . Pseudorehearsal techniques can be further classified depending on how the previous information is stored /used : • Knowledge distillation based approaches : An example method for lifelong learning using knowledge distillation ( Hinton et al. , 2015 ) is learning without forgetting ( LwF ) ( Li & Hoiem , 2017 ) - where distillation loss was added to match output of the updated network to that of the old network on the old task output variables . LwF and its extensions does not scale to a large number of new tasks , suffers from catastrophic forgetting and importantly do not address incremental learning or domain adapatation settings . • Rehearsal using exemplar sets : Rebuffi et al . ( 2017 ) propose iCaRL , an incremental learning approach by storing an exemplar set of data from the previous tasks and augmenting it with the new data . In more recent works , Javed & Paracha ( 2018 ) and Castro et al . ( 2018 ) have argued that a decoupled nearest mean classifier from Rebuffi et al . ( 2017 ) is not essential and have proposed joint learning of feature extraction and classification . Lopez-Paz et al . ( 2017 ) propose Gradient Episodic Memory , a technique which stores data from previous classes and constrains the gradient update while learning new tasks . • Rehearsal using generative models : Triki Rannen et al . ( 2017 ) propose to reproduce past knowledge by using task-specific under-complete autoencoders . When a new task is presented , optimization is constrained by preventing the reconstructions of these autoencoders from changing , thereby ensuring that the features necessary for the previous tasks are not destroyed . Shin et al . ( 2017 ) ; He et al . ( 2018 ) ; Venkatesan et al . ( 2017 ) employ generative adversarial networks for recreating the history from previous tasks . • Network regularization strategies : Methods belonging to this class aim to identify and selectively modify parts of the neural network which are critical to remember past knowledge or by explicitly penalizing loss of performance on old tasks . Kirkpatrick et al . ( 2017 ) ; Liu et al . ( 2018 ) use the Fischer information matrix to identify the most crucial weights responsible for prediction of a given task and lower the learning rate for these tasks . Lee et al . ( 2017 ) propose a dynamically expanding network to increase the capacity for new tasks if the previous architecture is insufficient to represent the data . Our framework lies at the intersection of pseudorehearsal methods and progressive neural networks , with scope for judiciously utilizing extra capacity , while resisting catastrophic forgetting . 3 LIFE-LONG LEARNING VIA FEATURE TRANSFORMATIONS . Before we present the feature transformer method , we introduce the terminologies and notations . We consider training a deep neural network which classifies an input to one of the classes c ∈ [ C ] , { 1 , 2 , · · · , C } . We refer to the operation of classifying an input to a particular class as a task . To this end , the classifier is trained with training dataset ( X , Y ) , drawn from a joint distribution ( X , Y ) . We view the deep neural network , defined by the parameters ( θ , κ ) , as the composition of a feature extractor Φθ : X → F , and a classifier Ψκ Ψκ ◦Φθ : X → [ C ] , ( 1 ) whereX is the space of input data , and F is a space of low-dimensional feature vectors . We concisely denote the training of the neural network by TRAIN ( θ , κ ; D ) , which minimizes a loss function on training data D = ( X , Y ) and produces the network parameters ( θ , κ ) . Let us also define the set of all computed features on input F = Φθ ( X ) . 1 When the loss function only penalizes misclassification , the network is expected to learn only the class separation boundaries in the feature space . However , as we demonstrate experimentally , good separation of class specific features enables stable learning of representations , which directly has a bearing on the performance of life-long learning . Therefore , in all our training procedures , we also use a feature loss which promotes feature separation model loss = classification loss ( θ , κ ) + λ · feature-loss ( θ ) . ( 2 ) In the lifelong learning context , we denote the time varying aspect of the network , training data and the classes , by using the time symbol t ∈ N , { 0 , 1 , 2 , · · · } in superscript on these objects . Realistically , at any time t > 0 , the classifier encounters any of the following canonical situations : 1 . The number of classes remain the same for the classifier as at t − 1 . The model encounters new training data for a subset of classes , without change in their distribution : ∀t > 0 , C ( t ) = C ( t−1 ) , T ( t ) 1 ⊆ [ C ( t ) ] and ∀τ ∈ T ( t ) 1 , ( X ( t ) τ , Y ( t ) τ ) ∼ ( X ( t ) τ , Y ( t ) τ ) = ( X ( t−1 ) τ , Y ( t−1 ) τ ) . 2 . The number of classes remain the same for the classifier as at t − 1 . The model en- counters new training data from modified input distribution ( s ) , for a subset of classes : ∀t > 0 , C ( t ) = C ( t−1 ) , T ( t ) 2 ⊆ [ C ( t ) ] and ∀τ ∈ T ( t ) 2 , ( X ( t ) τ , Y ( t ) τ ) ∼ ( X ( t ) τ , Y ( t ) τ ) 6= ( X ( t−1 ) τ , Y ( t−1 ) τ ) . 3 . The model encounters new class ( es ) and corresponding new data : ∀t > 0 , C ( t ) > C ( t−1 ) , T ( t ) 3 = [ C ( t ) ] \ [ C ( t−1 ) ] and ∀τ ∈ T ( t ) 3 , ( X ( t ) τ , Y ( t ) τ ) ∼ ( X ( t ) τ , Y ( t ) τ ) . In the most generic scenario , a combination of all the three situations can occur at any time index t , with training data available for the classes in the set T ( t ) , T ( t ) 1 ∪ T ( t ) 2 ∪ T ( t ) 3 . However , it is important to note that at every index t , the classifier is trained to classify all the C ( t ) classes . 3.1 FEATURE TRANSFORMATION BY AUGMENTING NETWORK CAPACITY . At any time t − 1 , the classifier is optimized to classify all the classes [ C ( t−1 ) ] and also the set of features F ( t−1 ) are well separated according to classes . At t , when new training data D ( t ) = ∪τ∈T ( t ) ( X ( t ) τ , Y ( t ) τ ) is encountered , the features extracted using the previous feature extractor ∂F ( t ) = ∪τ∈T ( t ) ( Φθ ( t−1 ) ( X ( t ) τ ) ) , ( 3 ) are not guaranteed to be optimized for classifying the new data and new classes . In order to achieve good performance on new data and classes , we propose to change the feature representation at time t , just before the classification stage . We achieve this by defining a feature transformer Φ∆θ ( t ) : F ( t−1 ) → F ( t ) , ( 4 ) 1Though Φθ is a mapping which acts on individual vectors , we abuse the notation here by using it with sets . parameterized by ∆θ ( t ) , which maps any feature extracted by Φθ ( t−1 ) to a new representation . The new feature extractor is now given by Φθ ( t ) , Φ∆θ ( t ) ◦ Φθ ( t−1 ) , where θ ( t ) , θ ( t−1 ) ∪ ∆θ ( t ) . Practically , this is realized by augmenting the capacity of the feature extractor at each time t by using one or more fully connected layers2 . It is however possible that Φ∆θ ( t ) could be simply an identity transform and feature transformers learnt in previous episodes could be adapted for new data . This helps in controlling the growth of network capacity over time and this aspect of our work is described in section 4.5 . The feature transformer is trained , along with a new classifier layer , using the composite loss function of the form in equation 2 , by invoking TRAIN ( ∆θ ( t ) , κ ( t ) ; D ( t ) ) , with D ( t ) = ( ∂F ( t ) , Y ( t ) ) 3 . This ensures that the classifier performs well on the new data . However , strikingly , training a feature transformer at t does not involve changing the feature extractor Φθ ( t−1 ) at all , and this helps us in alleviating catastrophic forgetting by efficiently making use of already computed features F ( t−1 ) through a memory module .

Summary:  a method is presented for on-going adaptation to changes in data, task or domain distribution. The method is based on adding, at each timed step, an additional network module transforming the features from the previous to the new representation. Training for the new task/data at time t relies on re-training with all previous data, stored as intermediate features. The method is shown to provide better accuracy than naïve fine tuning, and slightly inferior to plain re-training with all the data.