Continual Learning using the SHDL Framework with Skewed Replay Distributions

1 INTRODUCTION . Agents interacting with multiple tasks within any environment are required to acquire , fine-tune , and transfer knowledge for optimal functioning . The current state-of-the-art deep agents can learn and retain knowledge well when trained on individual tasks but their performance on previously learned tasks abruptly degrades when trained for one or more new tasks . This phenomenon is called catastrophic forgetting and is a long-standing challenge for machine learning and neural networks and , consequently , for the development of artificial intelligence ( AI ) systems ( Hassabis et al . ( 2017 ) ; Thrun & Mitchell ( 1995 ) ) . McCloskey & Cohen ( 1989 ) suggested the underlying cause of forgetting to be the distributed shared representation of tasks via network weights . Therefore training the network towards a new objective can cause almost complete forgetting of former knowledge . Several techniques have been proposed to encode knowledge in non-overlapping representations or ; preserve or limit changes to the weights corresponding to the learned knowledge . Kortge ( 1990 ) attempted to encode knowledge for different tasks with minimal overlap via activation sharpening algorithms . Lew recorded the inputs orthogonal and attempted orthogonal encoding at all hidden layers ( McRae & Hetherington ( 1993 ) , French ( 1994 ) ) . Recently , encoding methods like maxout and dropout ( Goodfellow et al . ( 2013 ) ) and local winner-takes-all ( Srivastava et al . ( 2013 ) ) have been explored to create sparsified feature representations to minimize task-specific representation overlap . However , mammalian brains undergo only gradual systematic forgetting suggesting that shared representations may not be the root cause of the problem . Several recent approaches have developed strategies that slow down learning on network weights which are important for previously learnt tasks . Kirkpatrick et al . ( 2016 ) preserved weights which correlate with previously acquired knowledge by using a fisher information matrix based regularizer . Zenke et al . ( 2017 ) slowed down learned on weights important for the previous tasks by using path integrals of loss-derivatives . An extreme version of the above-mentioned methods are Progressive neural networks ( Rusu et al . ( 2016 ) ) and Pathnets ( Fernando et al . ( 2017 ) ) that directly freeze important pathways in neural networks . This eliminates forgetting altogether but requires growing the network after each task and can cause the architecture complexity to grow with the number of tasks . These methods give superior performance as compared to the sparse representations encoding methods ( detailed before ) but may not be explicitly targeting the cause of catastrophic forgetting . Neuroscientific evidence ( McClelland et al . ( 1995 ) , O ’ Neill et al . ( 2010 ) ) has inspired another set of methods which focus on the replay of previously seen samples along with new samples to avoid catastrophic forgetting . The replay methods usually maintain an episodic memory of fixed-size with exemplars which are either directly replayed while learning e.g . in iCaRL ( Rebuffi et al . ( 2017 ) ) or indirectly utilized to adapt future gradient updates to the network e.g . inGEM ( Lopez-Paz & Ranzaton ( 2017 ) ) to avoid forgetting on previously seen tasks . Robins , 2004 proposed another replay technique , known as the pseudo-pattern rehearsal , to preserve the learned mappings by uniformly sampling random inputs and their corresponding outputs from networks and replaying them along with new task samples . Despite the superior performance of these methods and their biological significance , choosing to store samples from previous tasks is challenging as a large working memory may be required ( Lucic et al . ( 2017 ) ) . Recently , few generative approaches have been proposed to overcome this issue as these methods allow for storage of the relevant samples from previous tasks with limited memory . However , these methods give inferior performance as compared to methods with large storage capacity . This paper proposes a training regime that uses the modular ScatterNet Hybrid Deep Learning ( SHDL ) network proposed by Singh et al ( Singh & Kingsbury ( 2017a ; 2018 ) ) to achieve lifelong learning over two phases . The proposed methodology can retain the learned knowledge over phase 1 with minimal forgetting while using this learned knowledge to rapidly learn ( forward transfer ) new knowledge over phase 2 . The architecture of the modular network is computationally efficient and can learn the relevant representations using relatively fewer labelled examples as compared to the end-to-end supervised deep networks . The contributions of the paper are as follows : • Modular Architecture : This work uses the modular ScatterNet Hybrid Deep Learning ( SHDL ) network proposed by Singh et al . ( Singh & Kingsbury ( 2017a ; 2018 ) ) which is composed of a handcrafted frontend , an unsupervised midsection , and a supervised backend . The network is computationally efficient architecture and is constructed by selecting the optimal number of filters in each layer of the network . The network can learn rapidly with fewer labelled examples . This is beneficial as it is difficult to obtain a large number of labelled examples for most applications . • Rapid Learning with Forward Transfer : In addition to the ability of the proposed training regime to retrain previously learned knowledge with minimal forgetting . The framework is also able to build on the previously learned knowledge to acquire new knowledge rapidly leading to faster learning . • Efficient Memory Storage with Naturally Skewed Distributions : Most of the work on lifelong learning has primarily assumed iid distribution for all classes during replay . This is less likely to be true for the human experience as the distribution of classes seems to be more skewed towards the current experience over the recent ones . This distribution of data replayed to the network in this work follows similar natural assumptions . This also allows for efficient memory storage as only a fraction of data for the previous tasks needs to be stored . The classification performance of the proposed training regime is evaluated on the CIFAR-100 dataset for two phases . Multiple experiments are conducted with different relative replay frequencies between the learned tasks ( phase-0 ) and the new tasks ( phase-1 ) . Section 2 of the paper briefly presents the proposed training regime along with the SHDL network . Section 3 presents the experimental results while Section 4 draws conclusions . 2 TRAINING REGIME WITH SCATTERNET HYBRID DEEP LEARNING ( SHDL ) FRAMEWORK . This section introduces the proposed training regime for continual learning with the ScatterNet Hybrid Deep Learning ( SHDL ) Framework over two phases . The section presents the mathematical formulation of SHDL framework along with the training details ( Fig . 1 ) . 2.1 SCATTERNET HYBRID DEEP LEARNING ( SHDL ) FRAMEWORK . The ScatterNet Hybrid Deep Learning ( SHDL ) Framework is composed of a handcrafted frontend , an unsupervised midsection , and a supervised backend . Each layer of the network is designed and optimized automatically that produces the desired computationally efficient architecture . The handcrafted features allow for the rapid learning of the unsupervised module as it is not necessary to wait for the first layer to learn edges as they are already present . The unsupervised learning module is quite advantageous as it allows the network to learn meaningful representation from relatively fewer labelled examples as compared to fully supervised networks ( Singh & Kingsbury ( 2017a ; 2018 ) ) . 2.1.1 HAND-CRAFTED MODULE : SCATTERNET . Handcrafted descriptors are extracted using the parametric log based two-level Dual-Tree Complex Wavelet Transform ( DTCWT ) ScatterNet ( Singh & Kingsbury ( 2017b ) ) that extracts relatively symmetric translation invariant low-level features at the first layer and more discriminative sparse features at second layer ( Bruna & Mallat ( 2013 ) ; Singh & Kingsbury ( 2016 ) ) . The extracted features are also dense over the scale as they are obtained by decomposing multi-resolution images obtained at 1.5 times ( R1 ) and twice ( R2 ) the size of the input image . This ScatterNet ( Singh & Kingsbury ( 2017b ) ) is chosen over Bruna and Mallat ’ s ( Bruna & Mallat ( 2013 ) ) due to its superior classification accuracy and computational efficiency . The DTCWT ScatterNet formulation is presented for an input signal ( x ) which may then be applied to each multi-resolution image . The features at the first layer are obtained by filtering the input signal x with dual-tree complex wavelets ψj , r at different scales ( j ) and six pre-defined orientations ( r ) fixed to 15◦ , 45◦ , 75◦ , 105◦ , 135◦ and 165◦ , as shown in Fig . 2 . A more translation invariant representa- tion is built by applying a point-wise L2 non-linearity ( complex modulus ) to the real and imaginary part of the filtered signal : U [ L1 ] = √ |x ? ψaλ1 | 2 + |x ? ψbλ1 | 2 ( 1 ) The parametric log transformation layer is applied on the oriented features , extracted at the first scale j = 1 with a parameter kL1 [ j ] , to reduce the effect of outliers by introducing relative symmetry ( rs ) to their amplitude distribution ( as shown in Fig . 3 ) : U1rs [ j ] = log ( U [ L1 ] [ j ] + kL1 [ j ] ) , U [ L1 ] [ j ] = |x ? ψj | , ( 2 ) The parameter kL1 is selected such that it minimizes the difference between the mean and median of the distribution Singh & Kingsbury ( 2017b ) . Next , a local average is computed on the envelope |U1 [ λL1 ] | that aggregates the coefficients to generate the desired translation-invariant representation : Srs [ L1 ] = |U1rs| ? φ2J ( 3 ) The energy ( high-frequency components ) lost due to smoothing is recovered by cascaded wavelet filtering applied at the second layer Bruna & Mallat ( 2013 ) . The recovered components are again not translation invariant so invariance is achieved by first applying the L2 non-linearity to obtain the regular envelope : U [ L2 ] = |U1rs ? ψλL2 | ( 4 ) The parametric log transformation is applied again to produce relative symmetry : U2rs [ j ] = log ( U [ L2 ] [ j ] + kL2 [ j ] ) ( 5 ) Next , a local-smoothing operator is applied to improve translation invariance : : Srs [ L2 ] = U2rs ? φ2J ( 6 ) The output coefficients are typically formed from x ? φ ( Layer 0 ) , Srs [ L1 ] ( Layer 1 ) and Srs [ L2 ] ( Layer 2 ) for each of the two image resolutions R1 and R2 . 2.1.2 UNSUPERVISED LEARNING MODULE : CONVOLUTIONAL AUTOENCODER . The Scattering features extracted at ( L0 , L1 , L2 ) are concatenated and given as input to 4 stacked convolutional Autoencoder ( MasciUeli et al . ( 2011 ) ) . An autoencoder is a neural network that is trained to copy its input to its output . Internally , it has a hidden layer h that describes a code used to represent the input . The network may be viewed as consisting of two parts : an encoder function h = f ( x ) and a decoder that produces a reconstruction r = g ( h ) . The autoencoder learns useful information in the hidden layer h as it is constrained to have a smaller dimension than x ( undercomplete ) . Learning an undercomplete representation forces the autoencoder to capture the most salient features of the training data . The learning process is described simply as minimizing a loss function Lauto ( x , g ( f ( x ) ) ) ( 7 ) where Lauto is a loss function penalizing g ( f ( x ) ) for being dissimilar from x , such as the mean squared error .

Summary: The author proposed a modular SHDL with skewed replay distributions to do continual learning and demonstrated the effectiveness of their model on the CIFAR-100 dataset. They made contributions in three aspects: (1) using a computationally efficient architecture SHDL which can learn rapidly with fewer labeled examples. (2) In addition to retain previous learned knowledge, the model is able to acquire new knowledge rapidly which leads to fast learning. (3) By adopting the naturally skewed distributions, the model has the advantage of efficient memory storing.