Title: CONTINUAL LEARNING USING THE SHDL FRAME-WORK WITH SKEWED REPLAY DISTRIBUTIONS Anonymous

Abstract: Human and animals continuously acquire, adapt as well as transfer knowledge throughout their lifespan. The ability to learn continuously is crucial for the effective functioning of agents interacting with the real world and processing continuous streams of information. Continuous learning has been a long-standing challenge for neural networks as the repeated acquisition of information from nonuniform data distributions generally lead to catastrophic forgetting or interference. This work proposes a modular architecture capable of continuous acquisition of tasks while averting catastrophic forgetting. Specifically, our contributions are: (i) Efficient Architecture: a modular architecture emulating the visual cortex that can learn meaningful representations with limited labelled examples, (ii) Knowledge Retention: retention of learned knowledge via limited replay of past experiences, (iii) Forward Transfer: efficient and relatively faster learning on new tasks, and (iv) Naturally Skewed Distributions: The learning in the above-mentioned claims is performed on non-uniform data distributions which better represent the natural statistics of our ongoing experience. Several experiments that substantiate the above-mentioned claims are demonstrated on the CIFAR-100 dataset.

1. INTRODUCTION:

Agents interacting with multiple tasks within any environment are required to acquire, fine-tune, and transfer knowledge for optimal functioning. The current state-of-the-art deep agents can learn and retain knowledge well when trained on individual tasks but their performance on previously learned tasks abruptly degrades when trained for one or more new tasks. This phenomenon is called catastrophic forgetting and is a long-standing challenge for machine learning and neural networks and, consequently, for the development of artificial intelligence (AI) systems (Hassabis et al. (2017) ; Thrun & Mitchell (1995) ). McCloskey & Cohen (1989) suggested the underlying cause of forgetting to be the distributed shared representation of tasks via network weights. Therefore training the network towards a new objective can cause almost complete forgetting of former knowledge. Several techniques have been proposed to encode knowledge in non-overlapping representations or; preserve or limit changes to the weights corresponding to the learned knowledge. Kortge (1990) attempted to encode knowledge for different tasks with minimal overlap via activation sharpening algorithms. Lew recorded the inputs orthogonal and attempted orthogonal encoding at all hidden layers (McRae & Hetherington (1993) , French (1994) ). Recently, encoding methods like maxout and dropout (Goodfellow et al. (2013) ) and local winner-takes-all (Srivastava et al. (2013) ) have been explored to create sparsified feature representations to minimize task-specific representation overlap.

However, mammalian brains undergo only gradual systematic forgetting suggesting that shared representations may not be the root cause of the problem. Several recent approaches have developed strategies that slow down learning on network weights which are important for previously learnt tasks. Kirkpatrick et al. (2016) preserved weights which correlate with previously acquired knowledge by using a fisher information matrix based regularizer. Zenke et al. (2017) slowed down learned on weights important for the previous tasks by using path integrals of loss-derivatives. An extreme version of the above-mentioned methods are Progressive neural networks (Rusu et al. (2016) ) and Pathnets (Fernando et al. (2017) ) that directly freeze important pathways in neural networks. This eliminates forgetting altogether but requires growing the network after each task and can cause the architecture complexity to grow with the number of tasks. These methods give superior performance as compared to the sparse representations encoding methods (detailed before) but may not be explicitly targeting the cause of catastrophic forgetting.

Neuroscientific evidence (McClelland et al. (1995 ), O'Neill et al. (2010) ) has inspired another set of methods which focus on the replay of previously seen samples along with new samples to avoid catastrophic forgetting. The replay methods usually maintain an episodic memory of fixed-size with exemplars which are either directly replayed while learning e.g. in iCaRL (Rebuffi et al. (2017)) or indirectly utilized to adapt future gradient updates to the network e.g. inGEM (Lopez-Paz & Ranzaton (2017) ) to avoid forgetting on previously seen tasks. Robins, 2004 proposed another replay technique, known as the pseudo-pattern rehearsal, to preserve the learned mappings by uniformly sampling random inputs and their corresponding outputs from networks and replaying them along with new task samples.

Despite the superior performance of these methods and their biological significance, choosing to store samples from previous tasks is challenging as a large working memory may be required (Lucic et al. (2017) ). Recently, few generative approaches have been proposed to overcome this issue as these methods allow for storage of the relevant samples from previous tasks with limited memory. However, these methods give inferior performance as compared to methods with large storage capacity.

This paper proposes a training regime that uses the modular ScatterNet Hybrid Deep Learning (SHDL) network proposed by Singh et al (Singh & Kingsbury (2017a; 2018) ) to achieve lifelong learning over two phases. The proposed methodology can retain the learned knowledge over phase 1 with minimal forgetting while using this learned knowledge to rapidly learn (forward transfer) new knowledge over phase 2. The architecture of the modular network is computationally efficient and can learn the relevant representations using relatively fewer labelled examples as compared to the end-to-end supervised deep networks.

The contributions of the paper are as follows:

• Modular Architecture: This work uses the modular ScatterNet Hybrid Deep Learning (SHDL) network proposed by Singh et al. (Singh & Kingsbury (2017a; 2018) ) which is composed of a handcrafted frontend, an unsupervised midsection, and a supervised backend. The network is computationally efficient architecture and is constructed by selecting the optimal number of filters in each layer of the network. The network can learn rapidly with fewer labelled examples. This is beneficial as it is difficult to obtain a large number of labelled examples for most applications.

• Rapid Learning with Forward Transfer: In addition to the ability of the proposed training regime to retrain previously learned knowledge with minimal forgetting. The framework is also able to build on the previously learned knowledge to acquire new knowledge rapidly leading to faster learning.

• Efficient Memory Storage with Naturally Skewed Distributions: Most of the work on lifelong learning has primarily assumed iid distribution for all classes during replay. This is less likely to be true for the human experience as the distribution of classes seems to be more skewed towards the current experience over the recent ones. This distribution of data replayed to the network in this work follows similar natural assumptions. This also allows for efficient memory storage as only a fraction of data for the previous tasks needs to be stored.

The classification performance of the proposed training regime is evaluated on the CIFAR-100 dataset for two phases. Multiple experiments are conducted with different relative replay frequencies between the learned tasks (phase-0) and the new tasks (phase-1).

Section 2 of the paper briefly presents the proposed training regime along with the SHDL network. Section 3 presents the experimental results while Section 4 draws conclusions. 

2.1. SCATTERNET HYBRID DEEP LEARNING (SHDL) FRAMEWORK:

The ScatterNet Hybrid Deep Learning (SHDL) Framework is composed of a handcrafted frontend, an unsupervised midsection, and a supervised backend. Each layer of the network is designed and optimized automatically that produces the desired computationally efficient architecture. The handcrafted features allow for the rapid learning of the unsupervised module as it is not necessary to wait for the first layer to learn edges as they are already present. The unsupervised learning module is quite advantageous as it allows the network to learn meaningful representation from relatively fewer labelled examples as compared to fully supervised networks (Singh & Kingsbury (2017a; 2018) ).

2.1.1. HAND-CRAFTED MODULE: SCATTERNET:

Handcrafted descriptors are extracted using the parametric log based two-level Dual-Tree Complex Wavelet Transform (DTCWT) ScatterNet ( Singh & Kingsbury (2017b) ) that extracts relatively symmetric translation invariant low-level features at the first layer and more discriminative sparse features at second layer (Bruna & Mallat (2013) ; Singh & Kingsbury (2016) ). The extracted features are also dense over the scale as they are obtained by decomposing multi-resolution images obtained at 1.5 times (R1) and twice (R2) the size of the input image. This ScatterNet (Singh & Kingsbury (2017b) ) is chosen over Bruna and Mallat's (Bruna & Mallat (2013) ) due to its superior classification accuracy and computational efficiency. The DTCWT ScatterNet formulation is presented for an input signal (x) which may then be applied to each multi-resolution image.

The features at the first layer are obtained by filtering the input signal x with dual-tree complex wavelets ψ j,r at different scales (j) and six pre-defined orientations (r) fixed to 15 • , 45 • , 75 • , 105 • , 135 • and 165 • , as shown in Fig. 2 . A more translation invariant representa-tion is built by applying a point-wise L 2 non-linearity (complex modulus) to the real and imaginary part of the filtered signal:

U [L1] = |x ψ a λ1 | 2 + |x ψ b λ1 | 2 (1)

The parametric log transformation layer is applied on the oriented features, extracted at the first scale j = 1 with a parameter k L1 [j], to reduce the effect of outliers by introducing relative symmetry (rs) to their amplitude distribution (as shown in Fig. 3 ):

EQUATION

The parameter k L1 is selected such that it minimizes the difference between the mean and median of the distribution Singh & Kingsbury (2017b) . Next, a local average is computed on the envelope |U 1[λ L1 ]| that aggregates the coefficients to generate the desired translation-invariant representation:

S rs [L1] = |U 1 rs | φ 2 J (3)

The energy (high-frequency components) lost due to smoothing is recovered by cascaded wavelet filtering applied at the second layer Bruna & Mallat (2013) . The recovered components are again not translation invariant so invariance is achieved by first applying the L 2 non-linearity to obtain the regular envelope:

EQUATION

The parametric log transformation is applied again to produce relative symmetry:

EQUATION

Next, a local-smoothing operator is applied to improve translation invariance::

S rs [L2] = U 2 rs φ 2 J (6)

The output coefficients are typically formed from x φ (Layer 0), S rs [L1] (Layer 1) and S rs [L2] (Layer 2) for each of the two image resolutions R1 and R2.

2.1.2. UNSUPERVISED LEARNING MODULE: CONVOLUTIONAL AUTOENCODER:

The Scattering features extracted at (L0, L1, L2) are concatenated and given as input to 4 stacked convolutional Autoencoder (MasciUeli et al. (2011) ). An autoencoder is a neural network that is trained to copy its input to its output. Internally, it has a hidden layer h that describes a code used to represent the input. The network may be viewed as consisting of two parts: an encoder function h = f (x) and a decoder that produces a reconstruction r = g(h).

The autoencoder learns useful information in the hidden layer h as it is constrained to have a smaller dimension than x (undercomplete). Learning an undercomplete representation forces the autoencoder to capture the most salient features of the training data.

The learning process is described simply as minimizing a loss function

EQUATION

where L auto is a loss function penalizing g(f (x)) for being dissimilar from x, such as the mean squared error.

2.1.3. SUPERVISED LEARNING MODULE: PERCEPTRON:

The features learned by the hidden layer of the autoencoder are used by a perceptron (Rosenblatt (1958) ) to learn high-level features. The perceptron is trained using the binary cross entropy (BCE) with logits that is defined as:

L perc = - 1 N N -1 i=0 (y i log σ(ŷ i ) + (1 -y i ) log(1 -log σ(ŷ i ))) (8)

where L perc is a loss function computed on N samples with y as the ground truth label and ŷ is the prediction of the output layer of the perceptron.

2.2. PROPOSED TRAINING REGIME FOR CONTINUAL LEARNING:

The proposed training regime is used with the SHDL framework for continual learning over two phases. In the first phase (Phase-0), the SHDL framework is trained with the initial set of classes while in the second phase (Phase-1), the framework is trained with new classes and only a fraction of items from the Phase-0 classes. Since only a fraction of data is required to be stored, this leads to efficient memory storage. Also, knowledge learned during the first phase aids the learning of phase-1 classes.

Only the autoencoder and percepton modules of the framework are trained as the front-end is handcrafted. The autoencoder is jointly trained from scratch for classes of both phases to learn mid-level features. The module is trained on the scatternet features and is not evolved once it is trained. The percepton, on the other hand, is trained in two phases. In Phase-0, the output layer of the perceptron is composed of nodes (green) equivalent to the number of classes and initialized randomly, as shown in Fig. 1 . In phase-1, new classes are introduced to the framework corresponding to which new nodes (orange) are added in the output layer and again initialised randomly. However, the weights of all three layers of the perceptron, learned for the phase-1 classes are carried forward. The phase-1 training is performed with new classes along with a fraction of items from the classes introduced in phase-0. The weights carried forward for the first phase (Phase-0) allows the network to retain the performance on them even with the only fraction of the samples. The perceptron is trained using the BCE with logits as stated above.

2.3. REPLAY WITH RELATIVE FREQUENCIES:

The data distribution introduced to the SHDL framework in phase-1 is skewed towards the data samples for the new classes. This skewed distribution is constructed by sampling the items for the known classes relative to the new classes using the replay parameter (S p ) that is defined below:

EQUATION

where Known items corresponds to the items of the known (phase-0) classes while N ew items are items of the new (phase-1) classes. As an example, a 0.5 sampling parameter would correspond to 1 old item per 2 new items in the distribution.

2.4. PERFORMANCE MEASURES FOR CONTINUAL LEARNING:

The performance of the continual learning system is measured on the ability of the system to effectively learn new knowledge (phase-1) while being able to retain the previously learned knowledge (phase-0).

The performance on the new knowledge is measured using the (i) final performance and, (ii) the rate of learning while the retention on the learned knowledge in phase-0 is measured on the (iii) final performance and, (iv) time to final performance. The measures for the both learned and new knowledge are detailed below:

The performance measures for the phase-1 knowledge acquired by the network are detailed below:

• Performance gained: The system should reach similar or higher asymptotic performance compared to the baseline for new (phase-1) classes. This is measured as detailed below:

P g = Acc phase-1(new)

Acc baseline (10) where P g corresponds to the performance gained (P g >1) or lost (P g < 1) by the system for both new or known classes. For the ideal case, P g should be greater then or equal to one.

• Rate of learning: The knowledge learned during phase-0 should ideally aid the learning of phase-1 classes. This is measured using slope gained and area gained as detailed below:

-Slope Gained: The slope gained represents the faster acquisition of new knowledge is measured as detailed below: 11) where S g represents the slope gained. S corresponds to the slope of the learning curves obtained by fitting a sigmoid curve (dotted black curve) to them as shown in Fig. 2 . R > 1 represents faster learning as compared to the baseline. -Area Gained: Area gained by the phase-1 curve relative to the baseline is also used to measure the faster acquisition of new knowledge as shown below:

EQUATION

EQUATION

)

where A g corresponds to the area gained until convergence. A phase-1(new) represents the area under the curve for the new classes while A baseline represents area under the curve for the baseline curve. A g > 1 would represents positive area gained representing faster learning relative to the baseline.

The performance measures for the retention of phase-0 knowledge are detailed below:

• Performance gained: The system should reach similar close to the asymptotic baseline performance and is measure as shown below:

EQUATION

)

where P g corresponds to the performance gained (P g >1) or lost (P g < 1) by the system. For the ideal case, P g should be greater then or equal to one. • Time to final Performance: The performance of the learned knowledge during phase-0 gets interfered when new knowledge is introduced to the network. The performance on the learned tasks is gradually restored to a new asymptotic performance. The time to final performance (T F P old ) should ideally be immediate so smaller the time to the restoration better is the model.

3. EXPERIMENTAL RESULTS:

The performance of the proposed training regime is evaluated on the SHDL framework with several experiments conducted on the CIFAR-100 dataset (Krizhevsky & Hinton (2009) ). The experiments are constructed based on the relative frequencies between the new and known classes as well as the nature of the nature classes. The performance is measured based on the measures defined as detailed above. The details of the experiments as well as the training details of each section are presented below.

3.1. EXPERIMENTS ON CIFAR-100 DATASET:

The CIFAR-100 (Krizhevsky & Hinton (2009) ) dataset is constructed of 100 classes which are composed of 20 classes with each class further divided into 5 sub-classes. Each class consists of 500 training and 100 test images with each image of size 32 × 32 × 3.

The training regime as detailed consists of phase-0 where 15 classes sampled from the CIFAR-100 are introduced to the SHDL framework. In the second phase, 5 new classes are presented to the framework. Phase-0 consists of 500 examples for each class while in phase-1, the items in the 15 phase-0 classes are selected with frequency relative to the 5 phase-1 classes. The relative frequencies with which the known items are sampled are 0.0625, 0.125, 0.25, 0.50, 1.0. The 0.5 frequency means that the phase-1 distribution consists of 500 × 5 items for 5 new classes and 250 × 15 items for the 15 new classes.

The full test set of 10000 images is used for evaluation on all the experiments.

Four set (set-1 to set-4) of experiments are constructed by randomly selecting 15 phase-0 classes and then the corresponding 5 phase-1 classes.

3.2. SHDL TRAINING DETAILS:

The SHDL framework consists of a scatternet frontend, the autoencoder midsection and a supervised perceptron as the backend.

The frontend scatternet features are extracted from the input RGB image using DTCWT filters at 2 scales and 6 fixed orientations. Next, log transformation with parameter k j = 1.1 is applied to the representations obtained at the finer scale to introduce relative symmetry (Section. 2.1).

The convolutional autoencoder is composed of 4 stacked layers with layers that learns 256, 128, 64, and 32 filters respectively. The autoencoder is trained using SGD with a learning rate of 0.001 with a batch size of 32. The 32 feature map of size 4 × 4 are obtained at the hidden layers which are flattened to produce a feature vector of size 512 (32 × 4 × 4).

The 512 dimensional vector is given as input to the 3 layers perceptron with 128, 64, 15 or 20 (phase-0/phase-1) nodes per layer. The perceptron is trained to minimize the BCE loss using SGD with a learning rate of 0.001 and a batch size of 16. 

3.3. EVALUATION ON THE PROPOSED MEASURES:

The performance of the proposed training regime on different measures is shown in Table . 1 and Fig. 2 . The x axis of the graphs in Fig. 2 represents the number of iterations. The performance measure at each iteration is obtained by averaging the performance measures over 10 batches of 16 items each.

As observed from the table, the slope (S g ), area (A g ), and performance (P g ) is gained for new (phase-1) knowledge (>1) for both near and far experiments corresponding to all 4 set of experiment, when a minimum of 4 new item are replayed for each old item (R p ≤ 0.25). The performance on the known (phase-0) knowledge is lost (P g < 1) with R p ≤ 0.25 and requires replay for more than 1000 (T F P old ) iterations to reach final performance.

As more data samples for the learned knowledge are replayed (R p ≥ 0.50), the phase-0 performance is recovered much more rapidly (T F P old ≤ 1000) and with no or minimal performance loss. However, it gets much harder to learn new (phase-1) knowledge as indicated by S g < 1. This is also shown in Fig. 2 Table . 1 indicates that the he ideal scenario to learn phase-1 knowledge rapidly with minimal loss to the knowledge learned during phase-0 is when 4 new item are replayed for each old item (R p ≤ 0.25). This also leads to efficient memory storage as fewer samples from the old classes are required to be stored. EWC (Kirkpatrick et al. (2016) ) is designed to avoid catastrophic forgetting by regulating the loss function while iCARL (Rebuffi et al. (2017) ) is a class-incremental learner that classifies using a nearest-exemplar algorithm, and prevents catastrophic forgetting by using an episodic memory. The performance of the proposed method and the state-of-the-art is evaluated on the proposed measures and shown in Fig. 3 and Table 2 . The proposed method is able to outperform both EWC and iCARL.

4. CONCLUSION:

This work proposes a modular SHDL architecture that is able to acquire new knowledge rapidly while averting catastrophic forgetting by replaying replay only a fraction of items from the previ-ously learned knowledge learning to memory efficient architecture. The performance of the proposed method is validated on several proposed measures. The proposed training regime is able to outperform two state-of-the-art methods on the proposed measures further establishing the usefulness of the proposed method.

:



Figures and Tables:

Figure 1: The illustration presents the training regime used for continuous learning over two phases with the SHDL framework composed of the handcrafted frontend, unsupervised midsection, and supervised backend. In phase-0, the network learns an initial set of 15 classes (green connections). In phase-1, 5 new classes are introduced to the network while continuing training of the old 15 classes at different relative frequencies (0.0625, 0.125, 0.25, 0.50, 1.0)

Figure 2: The illustration presents plots for new knowledge and retention for both near and far experiments. The new knowledge plots show the rapid learning of new classes during the second phase as compared to the baseline. The retention plots for the classes learned during the first phase shows retention with only transient interference. Orange curve denoted the phase-1 plots while the blue curve represents the phase-0 plots. The dotted black curve represents the sigmoid fit to curves for both phases. The x axis of the graphs represents the number of iterations. The performance measure at each iteration is obtained by averaging the performance measures over 10 batches of 16 items each.

Figure 3: The illustration presents the performance of the proposed SHDL framework against SOTA EWC and ICARL frameworks-Slope gained, Area gained, Performance gained (old (phase-0), new (phase-1)).

COMPARISON WITH THE STATE-OF-THE-ART The performance of the proposed training regime is compared with two state-of-the-art methods, Elastic Weight Consolidation (EWC) (Kirkpatrick et al. (2016)) and Incremental Classifier and Representation Learning (iCARL) (Rebuffi et al. (2017)).

Classification error (%) on the CIFAR-10 dataset for the original CNN architectures and their corresponding DTSCNN architectures.

Table content: <table><tr><td>S p</td><td/><td colspan="3">Near Category</td><td/><td/><td/><td colspan="2">Far Category</td><td/></tr><tr><td/><td/><td>New</td><td/><td/><td>Old</td><td/><td>New</td><td/><td/><td>Old</td></tr><tr><td>Set-1</td><td>S g</td><td>A g</td><td colspan="3">P new P old T F P old</td><td>S g</td><td>A g</td><td colspan="3">P new P old T F P old</td></tr><tr><td colspan="4">0.062 1.87 1.98 1.38</td><td>0.63</td><td>1356</td><td colspan="3">3.68 2.05 2.42</td><td>0.58</td><td>1348</td></tr><tr><td colspan="4">0.125 1.64 1.76 1.33</td><td>0.77</td><td>1210</td><td colspan="3">3.56 1.86 2.36</td><td>0.71</td><td>1121</td></tr><tr><td colspan="4">0.250 1.17 1.23 1.24</td><td>0.92</td><td>1008</td><td colspan="3">2.91 1.41 2.24</td><td>0.86</td><td>865</td></tr><tr><td colspan="4">0.500 0.82 1.09 1.12</td><td>1.03</td><td>656</td><td colspan="3">1.05 1.21 1.99</td><td>1.00</td><td>512</td></tr><tr><td colspan="4">1.000 0.38 0.96 0.94</td><td>1.11</td><td>316</td><td colspan="3">0.94 0.98 1.67</td><td>1.12</td><td>449</td></tr><tr><td>Set-2</td><td>S g</td><td>A g</td><td colspan="3">P new P old T F P old</td><td>S g</td><td>A g</td><td colspan="3">P new P old T F P old</td></tr><tr><td colspan="4">0.062 1.83 1.90 1.44</td><td>0.68</td><td>1248</td><td colspan="3">3.53 2.13 2.53</td><td>0.63</td><td>1367</td></tr><tr><td colspan="4">0.125 1.60 1.68 1.39</td><td>0.77</td><td>1215</td><td colspan="3">3.19 1.92 2.41</td><td>0.79</td><td>1148</td></tr><tr><td colspan="4">0.250 1.13 1.15 1.30</td><td>1.44</td><td>1108</td><td colspan="3">2.781 1.54 2.33</td><td>0.94</td><td>889</td></tr><tr><td colspan="4">0.500 0.78 1.01 1.18</td><td>1.58</td><td>766</td><td colspan="3">1.05 1.29 1.99</td><td>1.16</td><td>512</td></tr><tr><td colspan="4">1.000 0.34 0.88 0.99</td><td>0.27</td><td>343</td><td colspan="3">0.77 1.03 1.74</td><td>1.24</td><td>452</td></tr><tr><td>Set-3</td><td>S g</td><td>A g</td><td colspan="3">P new P old T F P old</td><td>S g</td><td>A g</td><td colspan="3">P new P old T F P old</td></tr><tr><td colspan="4">0.062 1.93 2.05 1.40</td><td>0.76</td><td>1287</td><td colspan="3">3.67 2.19 2.72</td><td>0.49</td><td>1321</td></tr><tr><td colspan="4">0.125 1.70 1.81 1.36</td><td>0.85</td><td>1262</td><td colspan="3">3.51 1.72 2.59</td><td>0.76</td><td>1108</td></tr><tr><td colspan="4">0.250 1.23 1.28 1.20</td><td>1.02</td><td>1178</td><td colspan="3">2.66 1.32 2.38</td><td>0.81</td><td>796</td></tr><tr><td colspan="4">0.500 0.88 1.14 1.12</td><td>1.13</td><td>796</td><td colspan="3">0.96 1.17 1.93</td><td>1.09</td><td>541</td></tr><tr><td colspan="4">1.000 0.44 1.01 0.87</td><td>1.25</td><td>436</td><td colspan="3">0.94 1.09 1.54</td><td>1.27</td><td>423</td></tr><tr><td>Set-4</td><td>S g</td><td>A g</td><td>P g</td><td>P g</td><td>T F P old</td><td>S g</td><td>A g</td><td>P g</td><td>P g</td><td>T F P old</td></tr><tr><td colspan="4">0.062 2.04 1.98 1.34</td><td>0.55</td><td>1378</td><td colspan="3">3.55 2.31 2.42</td><td>0.64</td><td>1375</td></tr><tr><td colspan="4">0.125 1.81 1.76 1.27</td><td>0.67</td><td>1234</td><td colspan="3">3.23 1.91 2.36</td><td>0.83</td><td>1097</td></tr><tr><td colspan="4">0.250 1.34 1.23 1.19</td><td>0.82</td><td>1076</td><td colspan="3">2.19 1.32 2.24</td><td>0.98</td><td>786</td></tr><tr><td colspan="4">0.500 0.99 1.09 1.05</td><td>1.10</td><td>867</td><td colspan="3">1.56 1.07 1.99</td><td>1.14</td><td>493</td></tr><tr><td colspan="4">1.000 0.55 0.96 0.91</td><td>1.19</td><td>398</td><td colspan="3">0.83 0.84 1.67</td><td>1.29</td><td>431</td></tr></table>

The number of iterations required to reach final performance for old knowledge presented for the proposed and two state-of-the-art methods, EWC(Kirkpatrick et al. (2016)) and iCARL(Rebuffi et al. (2017)o

Table content: <table><tr><td>R p</td><td colspan="2">0.0625 0.125 0.25</td><td>0.5</td><td>1.0</td></tr><tr><td>SHDL</td><td>1296</td><td colspan="3">1169 1057 651 459</td></tr><tr><td>EWC</td><td>1418</td><td colspan="3">1381 1239 987 789</td></tr><tr><td>iCARL</td><td>2215</td><td colspan="3">1794 1611 1211 909</td></tr></table>

Peer review (iclr2020):
I think there might be some interesting ideas in the work, but I think the authors somehow did not manage to position themselves well within the *recent* works on the topic or even with respect to what continual learning (CL) is understood to be in these recent works. E.g. CL is a generic learning problem, and most algorithms are generic (with a few caveats in terms of what information is available) in the sense that they can be applied regardless of task (be it RL, be it sequence modelling etc.). This work seems limited to image classification. The SHDL wavelets pre-processing, if I understood it, is specific for images and probably even there under some assumption (e.g natural images). The autoencoder (middle bit) is trained on all tasks the CL needs to face, if I understood the work correctly (phase 0 + phase 1). This potentially makes the CL problem much simpler because you are limiting yourself to the top layer only when dealing with CL, not the rest. Not to mention that I don*t understand the motivation of the autoencoder. I think ample results show that unsupervised learning fails in many instances to provide the right features and underperforms compared to learning discriminative features by just backproping from the cross entropy (discriminative loss) all the way down. The only instance I know of for doing this is in low data regime where there is no alternative. I think the modularity used needs to be better introduced. Why the autoencoder, why the first layer of wavelets? Is it for the benefit for CL? I can understand the wavelets, since they are not learnt. But the autoencoder? The autoencoder being trained on all data feels like a cheat. I think the citation of the perceptron a bit strange. Do you really use the original perceptrion from 58? Why? We have much better tools now !? I think the different metrics introduced are interesting and useful. Though you should somehow find common ground to existing works as well to ensure a point of comparison. In the results section I almost got lost. What is the final performance on Cifar. How does this compare to a model that is not trained in a CL regime? What loss do you get from the proposed parametrizaton? In the comparison with EWC and iCarl, there the whole model was dealing with the CL problem, right? (all intermediary layers). I*m actually surprised iCarl is not doing better (I expect it can do better than EWC). Maybe provide a few more information of hyperparam used for this comparison. Overall I think the paper is not ready for being published. Not without addressing these points: * role of modularity (if not CL -- then why? ; is the modular structure original or part of the previous works cited, e.g. where the wavelets are introduced and so forth) * better integration with recent literature; provide answers and settings that allow apple to apple comparison so one can easily understand where this approach falls; if the method is not meant for this *traditional settings and metrics* please still provide them, and then motivate why this regime is not interesting and explain better the regime the method is meant for * as it stands the work is light on the low level details; Hyper-params and other details are not carefully provided (maybe consider adding an appendix with all of these). I have doubts that the work is reproducible without these details.