Continual Learning using the SHDL Framework with Skewed Replay Distributions
Authors: Amarjot Singh
Jay McClelland
Venue:  (2020)
Keywords: Continual Learning
Catastrophic Forgetting
SHDL
CIFAR-100

Abstract: Human and animals continuously acquire, adapt as well as transfer knowledge throughout their lifespan. The ability to learn continuously is crucial for the effective functioning of agents interacting with the real world and processing continuous streams of information. Continuous learning has been a long-standing challenge for neural networks as the repeated acquisition of information from non-uniform data distributions generally lead to catastrophic forgetting or interference.  This work proposes a modular architecture capable of continuous acquisition of tasks while averting catastrophic forgetting.  Specifically, our contributions are: (i) Efficient Architecture: a modular architecture emulating the visual cortex that can learn meaningful representations with limited labelled examples, (ii) Knowledge Retention: retention of learned knowledge via limited replay of past experiences, (iii) Forward Transfer: efficient and relatively faster learning on new tasks, and (iv) Naturally Skewed Distributions: The learning  in the above-mentioned claims is performed on non-uniform data distributions which better represent the natural statistics of our ongoing experience. Several experiments that substantiate the above-mentioned claims are demonstrated on the CIFAR-100 dataset.

Under review as a conference paper at ICLR 2020
CONTINUAL LEARNING USING THE SHDL FRAME- WORK WITH SKEWED REPLAY DISTRIBUTIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Human and animals continuously acquire, adapt well transfer knowledge throughout their lifespan. The ability to learn continuously is crucial for the ef- fective functioning of agents interacting with the real world and processing con- tinuous streams of information. Continuous learning has been a long-standing challenge for neural networks as the repeated acquisition of information from non- uniform data distributions generally lead to catastrophic forgetting or interference. This work proposes modular architecture capable continuous acquisition of tasks while averting catastrophic forgetting. Specifically, our contributions are: (i) Efficient Architecture: modular architecture emulating the visual cortex that can learn meaningful representations with limited labelled examples, (ii) Knowledge Retention: retention of learned knowledge via limited replay of past experiences, (iii) Forward Transfer: efficient and relatively faster learning new tasks, and (iv) Naturally Skewed Distributions: The learning in the above-mentioned claims is performed non-uniform data distributions which better represent the natu- ral statistics of our ongoing experience. Several experiments that substantiate the above-mentioned claims are demonstrated on the CIFAR-100 dataset.
INTRODUCTION
Agents interacting with multiple tasks within any environment are required to acquire, fine-tune, and transfer knowledge for optimal functioning. The current sfate-of-the-art deep agents can learn and retain knowledge well when trained individual tasks but their performance on previously learned tasks abruptly degrades when trained for one or more new tasks. This phenomenon is called catastrophic forgetting and is a Long-standing challenge for machine learning and neural networks and, consequently, for the development artificial intelligence (AI) systems (Hassabis et al. (2017);
Thrun & Mitchell (195)).
McCloskey & Cohen 1989) suggested the underlying cause of forgetting to be the distributed shared
representation of tasks via network weights. Therefore training the network towards new objective can cause almost complete forgetting of former knowledge. Several techniques have been proposed encode knowledge in non-overlapping representations or; preserve limit changes the weights corresponding to the learned knowledge.
Kortge 1990) attempted to encode knowledge for different tasks with minimal overlap via activation sharpening algorithms. Lew recorded the inputs orthogonal and attempted orthogonal encoding at all hidden layers (MeRae & Hetherington (199)). French (1944)) Recently, encoding methods like maxout and dropout (Goodfellow et al. (2013) and local winner-takes-all (Srivastava et al. (2013) have been explored create sparsified feature representations minimize task saecispec representation overlap.
However, mammalian brains undergo only gradual systematic forgetting suggesting that shared rep- resentations may not be the root cause of the problem. Several recent approaches have developed strategies that slow down learning on network weights which are important for previously learnt tasks. Kirkpatrick et al. (2016) preserved weights which correlate with previously acquired knowl- edge by using a fisher information matrix based regularizer. Zenke et al. (2017) slowed down learned on weights important for the previous tasks by using path integrals of loss-derivatives. An extreme version of the bbooe-mentionee methods are Progressive neural networks (Rusu et al. (2016) and Pathnets (Fernando al. (2017) that directly freeze important pathways in neural networks. This

---

Under review as a conference paper at ICLR 2020
eliminates forgetting altogether but requires growing the network after each task and can cause the architecture complexity to grow with the number of tasks. These methods give superior perfor- mance as compared to the sparse representations encoding methods (detailed before) but may not be explicitly targeting the cause of catastrophic forgetting.
Neuroscientific evidence (McClelland et al. (1995), O'Neill al. (2010)) has inspired another set of methods which focus on the replay of previously seen samples along with new samples to avoid catastrophic forgetting. The replay methods usually maintain an episodic memory of fixed-size with exemplars which are either directly replayed while learning e.g. in iCaRL (Rebuffi et al. (2017) or indirectly utilized to adapt future gradient updates to the network e.g. inGEM (Lopez- Pazezz-aa & Ranzaton (2017)) to avoid forgetting on previously seen tasks. Robins, 2004 proposed another replay technique, known as the pseudo-patters rehearsal, to preserve the learned mappings by uniformly sampling random inputs and their corresponding outputs from networks and replaying them along with new task samples.
Despite the superior performance of these methods and their biological significance, choosing to store samples from previous tasks challenging as large working memory may be required (Lu- cic et al. (2017)). Recently, few generative approaches have been proposed to overcome this issue as these methods allow for storage of the relevant samples from previous tasks with limited mem- ory. However, these methods give inferior performance compared to methods with large storage capacity.
This paper proposes training regime that uses the modular ScatterNet Hybrid Deep Learning (SHDL) network proposed by Singh et al (Singh & Kingsbury (2017a) 2018)) to achieve lifelong learning over two phases. The proposed methodology can retain the learned knowledge over phase with minimal forgetting while using this learned knowledge to rapidly learn (forward transfer) new knowledge over phase 2. The architecture of the modular network computationally efficient and can learn the relevant representations using relatively fewer labelled examples as compared to the end-to-end supervised deep networks.
The contributions of the paper are as follows:
Modular Architecture: This work uses the modular ScatterNet Hybrid Deep Learning (SHDL) network proposed by Singh et al. (Singh & Kingsbury (2017a) 2018)) which is composed of handcrafted frontend, an unsupervised midsection, and a supervised back- end. The network is computationally efficient architecture and is constructed by selecting the optimal number of filters in each layer of the network. The network can learn rapidly with fewer labelled examples. This is beneficial as it is difficult to obtain large number of labelled examples for most applications.
Rapid Learning with Forward Transfer: In addition to the ability of the proposed training regime to retrain previously learned knowledge with minimal forgetting. The framework is also able to build on the previously learned knowledge acquire new knowledge rapidly leading to faster learning.
Efficientt Memory Storage with Naturally Skewed Distributions: Most of the work lifelong learning has primarily assumed iid distribution for all classes during replay. This is less likely to be true for the human experience as the distribution of classes seems to be more skewed towards the current experience over the recent ones. This distribution of data replayed to the network in this work follows similar natural assumptions. This also allows for efficient memory storage as only fraction of data for the previous tasks needs to be stored.
The classification performance of the proposed training regime is evaluated on the CIFAR-100 dataset for two phases. Multiple experiments are conducted with different relative replay frequencies between the learned tasks (phase-0) and the new tasks (phase-1).
Section 2 of the paper briefly presents the proposed training regime along with the SHDL network. Section 3 presents the experimental results while Section 4 draws conclusions.
2

---

Under review as a conference paper at ICLR 2020
Data Selection Policy
Autoencoder
Data
ScatterNet
Cortical Perceptron
Old Now 1/16, 1/8, 1/4, 1/2, 1/1
Old Task
Autoencoder
ScatterNet
Cortical Percaptron
Phase-1
New Task
Figure 1: The illustration presents the training regime used for continuous learning over two phases with the SHDL. framework composed of the handcrafted frontend, unsupervised midsection, and supervised backend. In phase-0, the network learns an initial set of 15 classes (green connections). In phase-1, 1, 5 new classes are introduced to the network while continuing training of the old 15 classes at different relative frequencies (0.0625, 0.125, 0.25, 0.50, 1.0)
TRAINING REGIME WITH SCATTERNET HYBRID DEEP LEARNING (SHDL) FRAMEWORK
This section introduces the proposed training regime for continual learning with the ScatterNet Hy- brid Deep Learning (SHDL) Framework over two phases. The section presents the mathematical formulation of SHDL framework along with the training details (Fig. 1).
2.1 SCATTERNET HYBRID DEEP LEARNING (SHDL) FRAMEWORK
The ScatterNet Hybrid Deep Learning (SHDL) Framework is composed of a handcrafted frontend, an unsupervised midsection, and supervised backend. Each layer of the network is designed and optimized automatically that produces the desired computationally efficient architecture. The hand- crafted features allow for the rapid learning of the unsupervised module as it is not necessary to wait for the first layer to learn edges as they are already present. The unsupervised learning module quite advantageous as it allows the network to learn meaningful representation from relatively fewer labelled examples as compared to fully supervised networks (Singh & Kingsbury 2017a 2018))
2.1.1 HAND-CRAFTED MODULE: SCATTERNET
Handcrafted descriptors are extracted using the parametric log based two-level Dual-Tree Com- plex Wavelet Transform (DTCWT) ScatterNet (Singh & Kingsbury (20176)) that extracts relatively symmetric translation invariant lowelevel features at the first layer and more diseriminative sparse features at second layer (Bruna & Mallat 2013); [ingh & Kingsbury (2016).. The extracted features are also dense over the scale as they are obtained by decomposing multi-resolution images obtained at 1.5 times (RI) and twice (R2) the size of the input image. This ScatterNet (Singh & Kingsbury (2017))) is chosen over Bruna and Mallat's (Bruna & Mallat (2013)) due to its superior classifica- tion accuracy and computational efficiency. The DTCWT ScatterNet formulation is presented for an input signal (x)) which may then be applied to each multi-resolltionn image.
The features at the first layer are obtained by filtering the input signal with dual-tree complex wavelets dyr at different scales (j) and six pre-defined orientations (r) fixed to 15 , 105%, 135 and 165", shown in Fig. 2. A more translation invariant representa-
3

---

Under review as a conference paper at ICLR 2020
tion is built by applying point-wise L2 non non-linearity (complex modulus) the real and imaginary part the filtered signal:
U[Li] = xx oi,, + 1 * *//
(I)
The parametric log transformation layer is applied on the oriented features, extracted at the first scale j = 1 with a parameter L41 [f], to reduce the effect of outliers by introducing relative symmetry (rs)
to their amplitude distribution (as shown in Fig. 3):
-1,+ - log(LL]] kLID)), U|LIiD] I u
The parameter 1y is selected such that it minimizes the difference between the mean and median of the distribution Singh & Kingsbury (2017b), Next, local average is computed the envelope [U][[ that aggregates the coefficients to generate the desired canslation-invaiian representation:
Sru [L1] |U,, d22
(3)
The energy (high-frequency components) lost due to smoothing is recovered by cascaded wavelet filtering applied at the second layer [rruna & Mallat (2013). The recovered components are again translation invariant so invariance is achieved by first applying the L2 non-linearity obtain the regular envelope:
U[L2] 1U1rs A/
(4)
The parametric log transformation applied again produce relative symmetry:
U2 [i] log(UU + kı,l/))
(5)
Next, a local-smoothing operator is applied to improve translation invariance:
Sr, U2,s 02.
The output coefficients are typically formed from æ @ (Layer 0), Sys [L1] (Layer and S,, [L]] (Layer 2) for each of the two image resolutions R1 and R2.
2.1.2 UNSUPERVISED LEARNING MODULE: CONVOLUTIONAL AUTOENCODER
The Scattering features extracted (LO, L1, L2) are concatenated and given input to 4 stacked convolutional Autoencoder (MasciUeli et al. (20IT)). An autoencoder is a neural network that is trained copy its input its output. Internally, it has a hidden layer h that describes code used to represent the input. The network may be viewed as consisting two parts: an encoder function (h).
= f(x and decoder that produces a reconstruction
The autoencoder learns useful information in the hidden layer h as it is constrained to have smaller dimension than (undercomplete). Learning undercomplete representation forces the autoen- coder to capture the most salient features of the training data.
The learning process is described simply as minimizing loss function
Lauto (x,
where Louto is a loss function penalizing g(f for being dissimilar from x, such as the mean squared error.
2.1.3 SUPERVISED LEARNING MODULE: PERCEPTRON
The features learned by the hidden layer of the autoencoder are used by a perceptron (Rosenblati (1958)) to learn high-level features. The perceptron is trained using the binary cross entropy (BCE) with logits that is defined
4

---

Under review as a conference paper at ICLR 2020
1 N N
L Lpere-
Σ (us log (ūi) ys) log(1 log (j:)))
where Lpero is a loss function computed on N samples with y as the ground truth label and ğ is the prediction of the output layer of the perceptron.
2.2 PROPOSED TRAINING REGIME FOR CONTINUAL LEARNING
The proposed training regime is used with the SHDL framework for continual learning over two phases. In the first phase (Phase-O), the SHDL framework is trained with the initial set of classes while in the second phase (Phase-1), the framework is trained with new classes and only fraction of items from the Phase-O classes. Since only a fraction of data required to be stored, this leads to efficient memory storage. Also, knowledge learned during the first phase aids the learning of phase-1 classes.
Only the autoencoder and percepton modules of the framework are trained the front-end is hand- crafted. The autoencoder jointly trained from scratch for classes of both phases to learn mid-level features. The module is trained the scatternet features and is not evolved once it is trained. The percepton, on the other hand, is trained in two phases. In Phase-O, the output layer of the perceptron composed of nodes (green) equivalent to the number of classes and initialized randomly, as shown in Fig. 1. In phase-1, 1, new classes are introduced to the framework corresponding to which new nodes (orange) are added in the output layer and again initialised randomly. However, the weights of all three layers of the perceptron, learned for the phase-1 classes are carried forward. The phase- 1 training performed with new classes along with a fraction of items from the classes introduced in phase-0. The weights carried forward for the first phase (Phase-0) allows the network to retain the performance on them even with the only fraction of the samples. The perceptron is trained using the BCE with logits stated above.
2.3 REPLAY WITH RELATIVE FREQUENCIES
The data distribution introduced to the SHDL framework in phase-1 is skewed towards the data samples for the new classes. This skewed distribution constructed by sampling the items for the known classes relative to the new classes using the replay parameter (Sp) that is defined below:
Known
R
N Newssems
where Knownheems corresponds to the items of the known (phase-0) classes while N Newtteme are items the new (phase-1) 1) classes. As an example, a 0.5 sampling parameter would correspond to old item per 2 new items in the distribution.
2.4 PERFORMANCE MEASURES FOR CONTINUAL LEARNING
The performance of the continual learning system is measured on the ability of the system to effec- tively learn new knowledge (phase-1) 1) while being able to retain the previously learned knowledge (phase-0).
The performance on the new knowledge is measured using the (i) final performance and, (ii) the rate of learning while the retention on the learned knowledge in phase-0 is measured on the (iii) final performance and, (iv) time to final performance. The measures for the both learned and new knowledge are detailed below:
The performance measures for the phase-1 knowledge acquired by the network are detailed below:
Performance gained: The system should reach similar or higher asymptotic performance
compared to the baseline for new (phase-1) classes. This is measured as detailed below:
Ace 1(new)
Pg
(10)
AcChaseline
5

---

Under review as a conference paper at ICLR 2020
where Pg corresponds to the performance gained (P, >) or lost (P, < 1) by the system for both new or known classes. For the ideal case, Pg should be greater then or equal to one.
Rate of learning: The knowledge learned during phase-0 should ideally aid the learning phase-1 classes. This measured using slope gained and area gained as detailed below:
Slope Gained: The slope gained represents the faster acquisition of new knowledge measured as detailed below:
Sg Sphase-1(new)
(11)
Sbaseline
where Sg represents the slope gained. S corresponds to the slope of the learning curves obtained by fitting sigmoid curve (dotted black curve) to them as shown in Fig. 2. R represents faster learning compared the baseline.
Area Gained: Area gained by the phase-1 curve relative the baseline is also used to measure the faster acquisition of new knowledge as shown below:
Aphase ((hase-1(nen))
Ag
(12)
Abascline
where A, corresponds to the area gained until convergence. Aphase Aphase-1(new) 1(new) represents the area under the curve for the new classes while Aboseline represents area under the curve for the baseline curve. Ag > 1 would represents positive area gained represent- ing faster learning relative to the baseline.
The performance measures for the retention of phase-0 knowledge are detailed below:
Performance gained: The system should reach similar close the asymptotic baseline performance and is measure as shown below:
Pg ACCphase- 1(old)
(13)
Accbaseline
where Po corresponds to the performance gained (P, >1) lost (Ps < 1) by the system. For the ideal case, Pg should greater then or equal to one.
Time final Performance: The performance of the learned knowledge during phase-0 gets interfered when new knowledge is introduced to the network. The performance the learned tasks gradually restored to a new asymptotic performance. The time to final performance (TPP)t should ideally be immediate so smaller the time to the restoration better is the model.
EXPERIMENTAL RESULTS
The performance of the proposed training regime is evaluated on the SHDL framework with several experiments conducted on the CIFAR-100 dataset (Krizhevsky & Hinton (2099). The experiments constructed based the relative frequencies between the new and known classes well the nature of the nature classes. The performance is measured based on the measures defined as detailed above. The details of the experiments well the training details of each section are presented below.
3.1 EXPERIMENTS ON CIFAR-100 DATASET
The CIFAR- 100 (Krizhessky Hinton (2099) dataset constructed 100 classes which are composed of 20 classes with each class further divided into 5 sub-classes. Each class consists of 500 training and 100 test images with each image size 32 x 32 x 3.
The training regime detailed consists of phase-O where 15 classes sampled from the CIFAR-100 are introduced to the SHDL framework. the second phase, 5 new classes are presented to the framework.

---

Under review as a conference paper at ICLR 2020
Table 1: Classification eтror (%) on the CIFAR-10 dataset for the original CNN architectures and their corresponding DTSCNN architectures.
Sp
Near Category
Far Category
New
Old
New
Old
Set-1
S
Ag
P new
Potd
TFP
5,
Ag
P new
Pata
TFP
0.062
1.87
1.98
1.38
0.63
1356
3.68
2.05
2.42
0.58
1348
0.125
1.64
1.76
1.33
0.77
1210
3.56
1.86
2.36
0.71
1121
0.250
1.17
1.23
1.24
0.92
1008
2.91
1.41
2.24
0.86
865
0.500
0.82
1.09
1.12
1.03
656
1.05
1.21
1.99
1.00
512
1.000
0.38
0.96
0.94
1.11
316
0.94
0.98
1.67
1.12
449
Set-2
S
Ag
new
Pold
TFPa otd
Sg
Ag
P new
Pord
TFPote Pota
0.062
1.83
1.90
1.44
0.68
1248
3.53
2.13
2.53
0.63
1367
0.125
1.60
1.68
1.39
0.77
1215
3.19
1.92
2.41
0.79
1148
0.250
1.13
1.15
1.30
1.44
1108
2.781
1.54
2.33
0.94
889
0.500
0.78
1.01
1.18
1.58
766
1.05
1.29
1.99
1.16
512
1.000
0.34
0.88
0.99
0.27
343
0.77
1.03
1.74
1.24
452
Sct-3
S
Ag
New
Pold
TFPa old
Sg
Ag
Pnew
Potd
TFPo
0.062
1.93
2.05
1.40
0.76
1287
3.67
2.19
2.72
0.49
1321
0.125
1.70
1.81
1.36
0.85
1262
3.51
1.72
2.59
0.76
1108
0.250
1.23
1.28
1.20
1.02
1178
2.66
1.32
2.38
0.81
796
0.500
0.88
1.14
1.12
1.13
796
0.96
1.17
1.93
1.09
541
1.000
0.44
1.01
0.87
1.25
436
0.94
1.09
1.54
1.27
423
Sct-4
S
Ag
Po
P.
TFPad
S,
Ag
P.
Pa
TFPat
0.062
2.04
1.98
1.34
0.55
1378
3.55
2.31
2.42
0.64
1375
0.125
1.81
1.76
1.27
0.67
1234
3.23
1.91
2.36
0.83
1097
0.250
1.34
1.23
1.19
0.82
1076
2.19
1.32
2.24
0.98
786
0.500
0.99
1.09
1.05
1.10
867
1.56
1.07
1.99
1.14
493
1.000
0.55
0.96
0.91
1.19
398
0.83
0.84
1.67
1.29
431
Phase-O consists of 500 examples for each class while in phase- 1, the items in the 15 phase-0 classes are selected with frequency relative to the 5 phase-1 classes. The relative frequencies with which the known items are sampled are 0.0625, 0.125, 0.25, 0.50, 1.0. The 0.5 frequency means that the phase-l distribution consists of 500 x 5 items for 5 new classes and 250 x 15 items for the 15 new classes.
The full test set of 10000 images used for evaluation on all the experiments.
Four (set-1 to set-4) of experiments are constructed by randomly selecting 15 phase-O classes and then the corresponding 5 phase-l classes.
3.2 SHDL TRAINING DETAILS
The SHDL framework consists of a scatternet frontend, the autoencoder midsection and a supervised perceptron the backend.
The frontend scatternet features are extracted from the input RGB image using DTCWT filters at scales and 6 fixed orientations. Next, log transformation with parameter k; = 1.1 is applied to the representations obtained at the finer scale to introduce relative symmetry (Section. 2.1).
The convolutional autoencoder is composed of 4 stacked layers with layers that learns 256, 128, 64, and 32 filters respectively. The autoencoder trained using SGD with learning rate of 0.001 with batch size of 32. The 32 feature map of size 4 x 4 obtained at the hidden layers which are flattened to produce a feature vector of size 512 (32 x x 4).
The 512 dimensional vector is given input to the 3 layers perceptron with 128. 64, 15 20 Johase-O/phasee 1) nodes per layer. The perceptron is trained to minimize the BCE loss using SGD with learning rate of 0.001 and batch size of 16.

---

Under review as a conference paper at ICLR 2020
Continual Training Training from Scratch
Baseline (Continual)
Category: Near (top), Far (bottom)
Baseline (Scratch)
0.062
0.125
0.25
0.50
1,0
1.0
0--00733
1.0
1.0
30046
1.0
1.0
00000
New Knowledge
0.5
0.5
0.0
0
1000
2000
0.0
1000
2000
0.0
1000
2000
0.0
1000
2000
0.0
1000
2000
1.0
1.0
1.0
0033
0083
00003
000000
0001
Py
0.5
0.5
0.5
0.5
0.5
2000
0.0
2000
2000
0.0
2030
0.0
1000
2000
1.0
1.0
1.0
1.0
1.0
P-lod
0.5
0.5
0.5
0.5
0.5
Retention
1000
2000
1000
2000
1000
2000
0.0
1000
2000
1000
2000
1.0
1.0
1.0
1.0
1.0
P-102-0333 0.853
P-I9t-0715 0715
kgt 1965
P-95-1000 1.000
P- 1:328
0.5
0.5
0.5
0.5
0.0
1000
2000
0.0
1000
2000
0.0
1000
2000
0.0
0
1000
2000
0.0
1000
2000
Figure 2: The illustration presents plots for new knowledge and retention for both near and far experiments. The new knowledge plots show the rapid learning of new classes during the second phase as compared to the baseline. The retention plots for the classes learned during the first phase shows retention with only transient interference. Orange curve denoted the phase-1 plots while the blue curve represents the phase-O plots. The dotted black curve represents the sigmoid to curves for both phases. The axis of the graphs represents the number of iterations. The performance measure at each iteration obtained by averaging the performance measures over 10 batches of 16 items each.
3.3 EVALUATION ON THE PROPOSED MEASURES
The performance of the proposed training regime on different measures is shown in Table. I and Fig. 2. The axis of the graphs in Fig. 2 represents the number of iterations. The performance measure at each iteration is obtained by averaging the performance measures over 10 batches of 16 items each.
As observed from the table, the slope (Sg), area (Ag), and performance (Pg) is gained for new (phase-1) knowledge (> for both near and far experiments corresponding to all 4 set of experiment, when a minimum of 4 new item are replayed for each old item (Rp S 0.25). The performance on the known (phase-0) knowledge is lost (P, < 1) with Rd S 0.25 and requires replay for more than 1000 (TF Para) iterations to reach final performance.
As more data samples for the learned knowledge are replayed (Rp 2 0.50), the phase-0 perfor- mance is recovered much more rapidly FPot 1000) and with no or minimal performance loss. However, it gets much harder to learn new (phase-1) 1) knowledge as indicated by S, 1. This is also shown in Fig. 2
Table. indicates that the he ideal scenario to learn phase-1 knowledge rapidly with minimal loss to the knowledge learned during phase-0 is when 4 new item are replayed for each old item (Rp < 0.25). This also leads to efficient memory storage as fewer samples from the old classes are required to be stored.
8

---

Under review as a conference paper at ICLR 2020
SSHDL
EWC
ICARL
20
175
18
1.5
1.6
Slepe Gained
125
Area Gained
1.00
0.75
10
050
08
0.0625
0.125
1000
0.0625
0.125
0.25 Iterations
1000
Iterations
144
11
1.0
12-
Performance Gained
0.9
0.8
0.6
0.5
0.6
0.0625
0.125
0.25 Iterations
0.5
1000
0.0625
0.125
0.25 Reeations
0.5
1000
Figure 3: The illustration presents the performance of the proposed SHDL framework against SOTA EWC and ICARL frameworks- Slope gained, Area gained, Performance gained (old (phase-0), new (phase-1)).
3.4 COMPARISON WITH THE STATE-OP-THE-ART
The performance of the proposed training regime is compared with two state-of-the-art methods, Elastic Weight Consolidation (EWC) (Kirkpatrick al. (2016) and Incremental Classifier and Rep- resentation Learning (iCARL) (Rebuffi et al. (2017)).
EWC (Kirkpatrick et al. (2016) designed to avoid catastrophic forgetting by regulating the loss function while iCARL (Rebuffi et al. (017)) is a class-incremental learner that classifies using nearest-exemplar algorithm, and prevents catastrophic forgetting by using an episodic memory.
The number of iterations required to reach final performance for old knowledge presented for the proposed and two state-of-the-an methods, EWC (Kirkpatrick et al. (2066) and iCARL (Rebuffi et al. (017 o
Table 2:
R
0.0625
0.125
0.25
0.5
1.0
SHDL
1296
1169
1057
651
459
EWC
1418
1381
1239
987
789
iCARL
2215
1794
1611
1211
909
The performance of the proposed method and the state-of-the-art is evaluated on the proposed mea- sures and shown in Fig. 3 and Table 2. The proposed method is able to outperform both EWC and iCARL.
4 CONCLUSION
This work proposes a modular SHDL architecture that is able to acquire new knowledge rapidly while averting catastrophic forgetting by replaying replay only fraction of items from the previ-

---

Under review as a conference paper at ICLR 2020
ously learned knowledge learning to memory efficient architecture. The performance of the pro- posed method validated on several proposed measures. The proposed training regime able to outperform two sfate-of-the-att methods on the proposed measures further establishing the useful- ness of the proposed method.
ACKNOWLEDGMENTS
This research project has been supported by the lifelong Learning Machines (L2M) program of the Defence Advanced Research Projects Agency (DARPA). The U.S. Government is authorized to produce and distribute reprints for governmental purposes notwithstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA, or the U.S. Govermment.
REFERENCES
Joan Bruna and Stephane Mallat. Invariant scattering convolution networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 152-112-8886, 2013.
Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A. Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. arXiv: 108734, 2017.
Robert M. French. Dynamically constraining connectionist networks to produce orthogo- distristt representations reduce catastrophic interference. Cognitive Science Society, 24: 335-340, 1994.
Ian Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, and Yoshua Bengio. Dy- distributed namically constraining connectionist networks to produce distrbuee representations to reduce catastrophic interference. arXiv:1302.4389 24:335-340. 2013.
Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matt Botvinick. Neuroscience-
inspired artificial intelligence. Neuron Review, 95, 2017.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardinsa, An- drei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Brabska-Barwinsa, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic for- getting in neural networks. urxiv:
2016.
Carolyn Scott Kortge. Episodic memory in connectionist networks. Cognitive Science Society, 15: 764-771, 1990.
Alex Krizhevsky and Geoff Hinton. Learning multiple layers of features from tiny images. 2009.
David Lopez-Paz and MarcAurelio Ranzaton. Gradient episodic memory for continual learning. Neural Information Processing Systems, 2017.
Mario Lucic, atthew Faulkner, Andreas Krause, and Dan Feldman. Training mixture models scale via coresets. arxiv: /70.008/ 10, 2017.
Jonathan MasciUeli, MeierDan Cirean, and Jrgen Schmidhuber. Stacked convolutional auto- encoders for hierarchical feature extraction. International Conference on Artificial Neural Net- works (ICANN). 2011.
James McClelland, Bruce McNaughton, and Randall OReilly. Why there are complemen- tary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102:419 457, 1995.
Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. The Psychology of Learning Motivation, 24:104 1989.
10

---

Under review as a conference paper at ICLR 2020
ken McRae and Phil Hetherington. Catastrophic interference is eliminated in pretrained networks. Cognitive Science Sociery, 24:723-728, 1993.
Joseph O' O'Neill, Barty Pleydell-Bouverts David Dupret, and Jozsef Csicsvari. Play it again: reacti- vation of waking experience and memory. Trends Neuroscience, 33:220-229, 2010.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. Computer Vision and Pattern Recognition, 2017.
Rosenblatt. The perceptron: probabilistic model for information storage and organization in the brain. Psychological Review, 65:386-400 1958.
Andrei A. Rusu, Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadselln. Progressive neural networks. NEE.:1606.04677 2016.
Amarjot Singh and Nick Kingsbury. Multi-resolution dual-tree wavelet scattering network for signal. International Conference Mathematics in Signal Processing, 2016.
Amarjot Singh and Nick Kingsbury. Scatternet hybrid deep learning (shdl) network for object clas- sification. International Workshop on Machine Learning for Signal Processing, 2017a.
Amarjot Singh and Nick Kingsbury, Dual-tree wavelet scattering network with parametric log trans- formation for object classification. International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017b.
Amarjot Singh and Nick Kingsbury. Generative scatternet hybrid deep learning (g-shdl) network with structural priors for semantic image segmentation. IEEE International Conference on Acous- fics, Speech and Signal Processing, 2018
Rupesh K. Srivastava, Jonathan Masci, Sohrob Kazerounian, Faustino Gomez, and J urgen Schmid- huber. Compete to compute advances in neural information processing systems. Cognitive Science Society, 201:23102318, 2013.
Sebastian Thrun and Tom Mitchell. Lifelong robot learning. Robotics and Autonomous Systems, 15: 25-46, 1995.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. International Conference on Machine Learning, 2017.


Reviews:
[{"review_id": "Hkgn9zNX9r", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The author proposed a modular SHDL with skewed replay distributions to do continual learning and demonstrated the effectiveness of their model on the CIFAR-100 dataset. They made contributions in three aspects: (1) using a computationally efficient architecture SHDL which can learn rapidly with fewer labeled examples. (2) In addition to retain previous learned knowledge, the model is able to acquire new knowledge rapidly which leads to fast learning. (3) By adopting the naturally skewed distributions, the model has the advantage of efficient memory storing. \n\nOverall, the paper should be rejected because \n(1)the author spent too much space to introduce the off-the-shelf SHDL model which should be put in the Appendix or referred directly. In other words, the author should explain more details about the “replay” mechanism in their model and show the advantage of choosing SHDL rather than other deep neural nets under the continual learning paradigm. \n(2)The comparison with other methods are too simple. When comparing with other methods, the author should introduce the parameter setting and the detailed training strategy. Otherwise, the evidence made in the experimental section is not convincing. Besides, the author should follow the evaluation paradigm used in other published papers to make a fairer comparison.\n(3)The author should carry out more discussion about which part of their model contributes the most to the continual learning. After reading the paper thoroughly, I am still unclear about it. \n\nThe paper has some imprecise part, here are a few:\n(1)The caption in Table 1 is too simple. More details should be add to explain the table. \n(2)What is the DTSCNN in Table 1?\n(3)What is the green connections in Figure 1?\n(4)In the second contribution “Rapid learning with forward transfer”, is the ability to “retrain” or “retain” the previous learned knowledge?", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HJgg5GG59B", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I think there might be some interesting ideas in the work, but I think the authors somehow did not manage to position themselves well within the *recent* works on the topic or even with respect to what continual learning (CL) is understood to be in these recent works. \n\nE.g. CL is a generic learning problem, and most algorithms are generic (with a few caveats in terms of what information is available) in the sense that they can be applied regardless of task (be it RL, be it sequence modelling etc.). This work seems limited to image classification. The SHDL wavelets pre-processing, if I understood it, is specific for images and probably even there under some assumption (e.g natural images). \n\nThe autoencoder (middle bit) is trained on all tasks the CL needs to face, if I understood the work correctly (phase 0 + phase 1). This potentially makes the CL problem much simpler because you are limiting yourself to the top layer only when dealing with CL, not the rest. Not to mention that I don't understand the motivation of the autoencoder. I think ample results show that unsupervised learning fails in many instances to provide the right features and underperforms compared to learning discriminative features by just backproping from the cross entropy (discriminative loss) all the way down. The only instance I know of for doing this is in low data regime where there is no alternative. \n\nI think the modularity used needs to be better introduced. Why the autoencoder, why the first layer of wavelets? Is it for the benefit for CL? I can understand the wavelets, since they are not learnt. But the autoencoder? The autoencoder being trained on all data feels like a cheat. \n\nI think the citation of the perceptron a bit strange. Do you really use the original perceptrion from 58? Why? We have much better tools now !?\n\nI think the different metrics introduced are interesting and useful. Though you should somehow find common ground to existing works as well to ensure a point of comparison. In the results section I almost got lost. What is the final performance on Cifar. How does this compare to a model that is not trained in a CL regime? What loss do you get from the proposed parametrizaton?\n\nIn the comparison with EWC and iCarl, there the whole model was dealing with the CL problem, right? (all intermediary layers). I'm actually surprised iCarl is not doing better (I expect it can do better than EWC). Maybe provide a few more information of hyperparam used for this comparison.\n\nOverall I think the paper is not ready for being published. Not without addressing these points:\n * role of modularity (if not CL -- then why? ; is the modular structure original or part of the previous works cited, e.g. where the wavelets are introduced and so forth)\n * better integration with recent literature; provide answers and settings that allow apple to apple comparison so one can easily understand where this approach falls; if the method is not meant for this \"traditional settings and metrics\" please still provide them, and then motivate why this regime is not interesting and explain better the regime the method is meant for\n * as it stands the work is light on the low level details; Hyper-params and other details are not carefully provided (maybe consider adding an appendix with all of these). I have doubts that the work is reproducible without these details. ", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper."}, "extra_text": {}}, {"review_id": "B1xG4i5TFH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper suggest to use the previously proposed ScatterNet Hybrid Deep Learning  (SHDL) network in a continual learning setting. This is motivated by the fact that the SHDL needs less supervised data, so keeping a small replay buffer can be enough to maintain performance while avoiding catastrophic forgetting.\n\nMy main doubt is the benchmark and evaluation of the proposed method. The metrics reported are all relative to a baseline value (which I could not find reported), and make it difficult to understand how the model is performing in absolute term. This is particularly a problem when comparing with existing state of the art method (Fig. 3, Table 4), since this does not exclude that they may have an overall much better accuracy in absolute terms.\n\nAlso concerning the comparison with the previous literature, I could find no details about the architecture and the training algorithm used. Notice that this may in particular affect some the reported metrics, since they depend on the shape of the training curve (reporting the training curves for all methods may also be useful). Also, since SHDL uses a small replay buffer, are EWC and the other method modified to use the replay buffer and make the comparison fair?\n\nWhile several standard tests for continual learning exists (for example the split CIFAR10/100 in Zenke et al., 2017), those are not used, and rather a simpler test is used which only attempt to learn continually two datasets.  It would be helpful to also report a direct comparison on those tests.\n\nRegarding the line: \"The autoencoder is jointly trained from scratch for classes of both phases to learn mid-level features\", does this mean that the auto-encoder is trained using data of the two distributions at the same time rather than one after the other? If it is the former case, while it is unsupervised training, it would be a deviation from the standard continual learning framework and should clearly be stated.\n\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper adapts a previously proposed modular deep network architecture (SHDL) for supervised learning in a continual learning setting.  One problem in this setting is catastrophic forgetting.  The proposed solution replays a small fraction of the data from old tasks to avoid forgetting, on top of a modular architecture that facilitates fast transfer when new tasks are added.  The method is developed for image inputs and evaluated experimentally on CIFAR-100.

The reviews were in agreement that this paper is not ready for publication.  All the reviews had concerns about the lack of explanation of the proposed solution and the experimental methods.  The reviewers were concerned about the choice of metrics not being comparable or justified: Reviewer4 wanted an apples-to-apples comparison, Reviewer1 suggested the paper follow the evaluation paradigm used in earlier papers, and Reviewer2 described the absence of an explained baseline value.  Two reviewers (Reviewer4 and Reviewer2) described the lack of details on the parameters, architecture, and training regime used for the experiments.  The paper did not not justify which aspects of the modular system contributed to the observed performance (Reviewer4 and Reviewer1).   Several additional concerns were also raised. 

The authors did not respond to any of the concerns raised by the reviewers. 

Decision: Reject