Continual Learning Using Task Conditional Neural Networks
Authors: Honglin Li
Frieder Ganz
David J. Sharp
Payam M. Barnaghi
Venue: ICLR 2022 Submitted (2022)
Keywords: catastrophic forgetting
continual learning

Abstract: Conventional deep learning models have limited capacity in learning multiple tasks sequentially. The issue of forgetting the previously learned tasks in continual learning is known as catastrophic forgetting or interference. When the input data or the goal of learning changes, a continual model will learn and adapt to the new status. However, the model will not remember or recognise any revisits to the previous states. This causes performance reduction and re-training curves in dealing with periodic or irregularly reoccurring changes in the data or goals. Dynamic approaches, which assign new neuron resources to the upcoming tasks, are introduced to address this issue. However, most of the dynamic methods need task information about the upcoming tasks during the inference phase to activate the corresponding neurons. To address this issue, we introduce Task Conditional Neural Network which allows the model to identify the task information automatically. The proposed model can continually learn and embed new tasks into the model without losing the information about previously learned tasks. We evaluate the proposed model combined with the mixture of experts approach on the MNIST and CIFAR100 datasets and show how it significantly improves the continual learning process without requiring task information in advance.

Under review as a conference paper at ICLR 2021
CONTINUAL LEARNING USING TASK CONDITIONAL NEURAL NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Conventional deep learning models have limited capacity in learning multiple tasks sequentially. The issue of forgetting the previously learned tasks in continual learning known as catastrophic forgetting or interference. When the input data or the goal of learning changes, a continual model will learn and adapt to the new status. However, the model will not remember recognise any revisits the previous states. This causes performance reduction and re-training curves in dealing with periodic or irregularly reoccurring changes in the data or goals. Dynamic approaches, which assign new neuron resources to the upcoming tasks, are introduced to address this issue. However, most of the dynamic methods need task information about the upcoming tasks during the inference phase to activate the corresponding neurons. To address this issue, we introduce Task Conditional Neural Network which allows the model to identify the task information automatically. The proposed model can continually learn and embed new tasks into the model without losing the information about previously learned tasks. We evaluate the proposed model combined with the mixture of experts approach on the MNIST and CIFARIOG datasets and show how it significantly improves the continual learning process without requiring task information in advance.
INTRODUCTION
The human brain can adapt and learn new knowledge in response to changing environments. Hoshi al. (1998) demonstrate that we can continually learn different tasks while retaining previously learned variations of the same similar phenomenon and give different reactions under different contexts. Similarly, Asaad et al. 2000) have found that our neurons are task-independent. Under different context, the neurons the machine learning models, in a scalable way, are not capable of adapting to changing environments quickly and automatically using artificial neurons corresponding different tasks. Consequently, conventional machine learning models tend to forget the previously learned task after learning a new task. This scenario is known as catastrophic forgetting interference in machine learning (McCloskey & Cohen (989)).
fired selectively with respect the stimulus. In contrast, most of
Catastrophic interference problem in machine learning is one of the hurdles to implement general artificial intelligence learning systems (Legg & Hutter (2007). Unable to learn several tasks, a model should be trained with all the possible scenarios in advance. This requirement intractable practice and is not inline with the continual learning models (Trun & Mitchell (1995).
Continual machine learning algorithms change over time and adapt their parameters to data or learning goals changes. We refer to the learning goal or specific part of the data with a learning goal as task. The learning models are not often equipped with solutions quickly adapt to the situations which they have seen before if their parameters have significantly changed over time by continual learning.
variety of continual learning methods have been proposed to solve the problems mentioned above. Shin et al. (2017) proposed memory-based approach which is replay the trained samples to solve the forgetting problem while learning a new task. Lee et al. (217)) y: Kirkpatrick et al. (2017): Zeng let al. (2018) leverage the regularisation methods reduce the representational overlap of different tasks. Dynamic network approaches assign extra neuron resources to new tasks (Yoon et al. (2017)) Most of the existing solutions need to know all the task changes in advance, this information is given

---

Under review as a conference paper at ICLR 2021
manually to the model throughout the learning. Some other works detect in-task and out-task samples to inference the task identity (Li et al. (2019); Lee et al. (2020)).
In this work, we propose novel method to overcome the catastrophic forgetting problem. We leverage Misture of Experts (MoE) (Masoudnia & Ebrahimpour (2014) to train the lask-specific models. One of the advantage of MoE is that can a shallower networks and utilise the capacity as much as possible for each expert. Theoretically, the potential capacity of the combined networks is infinite. In other words, we do need initialise giant network the beginning of the training to avoid learning further tasks without forgetting them. In each expert. we integrate the probabilistic neural networks (Specht (1990)) and conventional neural networks (Haykin (1994)) to produce the task likelihood. The final prediction of each expert will be weighted by the task likelihood and calculate the task conditional probabilities. We name our approach as Task Conditional Neural Network (TCNN).
2 RELATED WORK
There are different approaches address the forgetting problem in continual learning. Parisi et al.) (2019) categorise these approaches into three groups: Regularisation, Memory Replay and Dynamic Network approaches.
The regularisation approaches find the overlap of the parameter space between different tasks. One of the popular algorithms in this group is Elastic Weight Consolidation (EWC) proposed by Kirkpatrick et al. (201)). EWC avoids significantly changing the parameters that are important to learned task. assumes the weights have Gaussian distributions and approximates the posterior distribution of the weights by the Laplace approximation. A similar idea is used in Incremental Moment Matching (IMM) proposed by Lee et al. (2017). IMM finds the overlap of the parameter distributions by smoothing the loss surface of the tasks. Zeng et al. (2018) address the forgetting problem by allowing the weights to change within the same subspace of the previously learned task. Li & Hoiem| (2018)
address the problem by using the knowledge distillation (Hinton et al. (2015). They enforce the prediction of the learned tasks to be similar to the new tasks (Parisi et al. (2019)) However, these models require advance knowledge of the training tasks and the task changes.
Memory Replay methods mainly focus on interleaving the trained samples with the new tasks. A pseudo-rehearsal mechanism proposed by Robins (1995) reduce the memory requirement store the training samples for each task. In a pseudo-rehearsal. instead of explicitly storing the entire training samples, the training samples of previously learned tasks are drawn from a probabilistic distribution model. (Shin et al.) (0177) propose an architecture consisting of deep generative model and task solver. Similarly, Kamra al. (2017) use a variational autoencoder to generate the previously trained samples. However, this group of models are complex to train, and in real-world cases, the sampling methods do not offer an efficient solution for sporadic and rare occurrences. These models also often require advance knowledge of the change occurring.
Dynamic Networks allocate new neurons to new tasks. Yoon et al. (0177) propose Dynamic Ex-
pandable Networks (DEN) to learn new tasks with new parameters continuously. Similarly, Serrá et al. 2018) also allocate new parameters to learn new tasks. Masse et al. (2018) propose dynamic networks Context Dependent Gating (XdG) which provides the task context information to train the model. However, this group of models require the task information to be given to the model explicitly, In other words, the model knows in advance which neurons should be activated to perform each test task. To identify which neurons or experts should be used during the test state, AAljundi et al. (2017) proposed the Expert Gate and Lee al. (2020) proposed Continual Neural Dirichlet Process Mixture (CN-DPM) leverage generative models to achieve the task-free continual learning. which does not need the task identities the test phase. For each task, the Expert Gate and CN-DPM need to train a generator to distinguish the in-task and out-task samples. As the complexity of the dataset grows, the extra demand including parameters and computational resources of training a generator increases. Different from them, we introduce the probabilistic layer added into the discriminative model. Our method saves the memory since we do not need an extra model which is a generator to recognise the task identity. Furthermore, our approach save the computational resources (the probabilistic layer can be simultaneously trained with classification tasks and we do not need to train a generator for each expert).
2

---

Under review as a conference paper at ICLR 2021
van de Ven & Tolias (0119) define different scenarios to illustrate the demand of model for the task information. Generally, the class- innssmnncremtttll scenario (Class-IL) represents the model does not need the task information in advance, musk-incremental scenario (Task-IL) represents the model need the task information at inference phase.
3 TASK CONDITIONAL NEURAL NETWORK
In the TCNN, each expert is independent of others and has the same behaviour during the training and inference phases. One model contains multiple expert corresponding to a different task. this section, we introduce our model by taking one expert as example.
3.1 ESTIMATING THE TASK LIKELIIOOD
In this work, we refer to the task likelihood as P(t = ke[z)) to represent the sample r from the kes task. To estimate the task likelihood efficiently, we add probabilistic layer into expert. Different from the conventional neural networks, this model contains two heads performing estimating task likelihood and classification. The classification head is connected to a classification layer, which is similar conventional neural network. The task likelihood head is connected to the probabilistic layer. The probabilistic layers contains several kernels, each of which performs the function shown in Equation Where the f() is the function of hidden, 2; is vector representing the inh kernel, zj has the same dimension with the output from previous layer, 2 is the covariance matrix of 21:n. where n is the number of the kernels in the probabilistic layer. Calculating the covariance matrix could be intractable due to the high dimension of 21:m- Since the kernels can be viewed as different data patterns, we assume the kernels in the probabilistic layers are independent of each other.
k()(x), -xxp) -
z:)
(1)
The Equation T measures the similarity between the input and existing kernels acting as anchors. The output of the probabilistic layer an n dimensional vector. Each element in the vector can be regarded as the similarity of the input to one of the existing kernel. The summation of these similarities can be viewed as the task likelihood. To estimate the task likelihood, we perform normalised summation shown in Equation (2) The range of the task likelihood P(t(ar, is from 0 to
E (xf(x),
P((,,))
a Σ k(f(x) +1 max (K()(x),,,)) J=I...
While training new expert, we jointly maximise the task likelihood and the classification likelihood by minimizing Equation 3 Where {z, y is the training sample set of task k, C' is the number of
classes in the task k, (x((0) y (e)} is the samples of class, 0g is the parameters in the hidden layers shared by classification and probabilistic layer, OD and Or are the parameters in the classification and probabilistic layers respectively, À is hyper-parameter weight the task likelihood loss, N is number samples.
C
(.(0s,0p.Or)
= N (0) log (1)0)
((000)(0,,)))))) 0p) + x log P(t k/2, 0r,0s))
(3)
3.2 TRAINING AND INFERENCE
Each expert is corresponding to single task. We augment each samples to generate different views and make these views converge to the kernels in the probabilistic layer. A view generated by different pre-defined functions a variation the input. In this work, we define several functions to get a different view of the inputs. Including Zero Component Analysis (7CA) Pal & Sudeep (2016), shift,
3

---

Under review as a conference paper at ICLR 2021
rotation shear and flip. More specifically, we minimize the loss function 4 where m is the number of agumented views, (fs, 0D 0r) is the loss for the Ut view.
After training all K experts, we augment the test data by the same functions used in training process of each expert, and estimate the P(y(z) by Equation (5) where æ, is the Vth view after augmentation, t k represents the kec expert is responsible for the input. Overall, process by averaging several different views. The idea is inspired by contrastive learning (Chen et al.) (2020)). We take the in-task samples positive pairs and out-task samples negative pairs train a good feature extractor. However, in the continual learning scenarios, the out-task samples is not accessible. Hence we average multiple different views of the in-task samples to learn an anchor to distinguish the mn-task and out-task samples.
do the training and inference
T (Os, Dn 0T)
(4)
777 1
K
1 P(y|z) s m Σ P(y.
1 R
Plyolzv = k)P(t k|xx)
(5)
EXPERIMENTS AND EVALUATIONS
test our model on the Modified National Institute of Standards and Technology (MNIST) hand- written digits dataset. We also use the Canadian Institute For Advanced Research (CIFAR) 100 dataset, which is a collection of images. See Table [ We pre-define the covariance matrix in the probabilistic layers an identity matrix. The weight factor À in Equation ( is set to 0.1. We assume each class in each task can be represented by a single data pattern, and set the number of kernels in the probabilistic layer
as the number of classes in each task.
We compare our model with several state-of-thee approaches including memory replay approaches referring to Generative replay (GR) (Shin al. (2017)), Brian Inspired Replay (BI-R) (van de Ven iet al. (2020) and Learning without Forgetting (LwF) (Li & Hoiem (018)). regularisation approaches
including Synaptic Intelligence (SI) (Zenke al. (2017) and Elastic Weight Consolidation (EWC)
(Kirkpatrick et al. (2017). dynamic networks Context Dependent Gating (XdG) (Masse et al. (2018)). Besides that, we compare our method Expert Gate (Aljundi al. (2017) and CN-DPM (Lee et al. (2020)), both of which use the MoE architecture well. We test the methods in scenario (Class-IL.) for the methods except XdG. XdG can only perform in task-incremental scenario (Task-IL). In other words, the XdG is informed with task identity in the test phase. We also train two models as baselines. The joint represents offline training which trains a model with all the data at once. The None represents online training without any continual learning techniques.
Table 1: Dataset used in the experiments.
Dataset MNIST
Tasks 5
Classes 10
Classes per task
Train set
Test set
2
50K
10K
CIFARION 100
20
100
5
50K
10K
To show how much the model forgets the previously learned task, calculate the forgetting rate of t
task i after learning task T. We denote a as the test accuracy on the test set of task after learning task I' and compute the forgetting rate by Equation
max LEI.TT-
( dT
(6)
After learning task T, calculate the forgetting rate of all the previously learned tasks and show the forgetting as a lower triangular matrix. We refer this matrix as forgetting matrix to show how much the model forget the previously learned tasks during the training process.
4

---

Under review as a conference paper at ICLR 2021
TCNN
BI-R
1.0
0.030
0.08
Average Test Accuracy
3
0.06
0.8
0.018
0.012
0.04
0.006
0.00
0.8
0.000
0.00
GR
None
0.4
0.125
0,100
0.075
None EWCLLMM LWF
0.2
0.025
2
Number of Tasks
(a)
(b)
Figure 1: Split MNIST experiment. (a). Average test accuracy on the split MNIST experiment. All the approaches are tested in Class-IL scenario except XdG which performs Task-IL scenario. (b). Forgetting matrix of the model in the split MNIST experiment. In the forgetting matrix, the value at deh column and jeh row is the forgetting rate of task after learning task
4.1 SPLIT MNIST EXPERIMENT
The split MNIST experiment is a benchmark experiment in continual learning field (Zenke et al. (2017): Lee et al. 7) Masse et al. (2018)). We split the MNIST into different tasks of consecutive digits referring to 0/1, 2/3, 4/5, 6/7, 8/9. The basic architecture an expert is a convolutional neural network with two convolutional and fully-connected layers. We added probabilistic layer containing two kernels as the extra head. We use the Adam optimizer with a learning rate 0.001.
As shown in Figure Ta the proposed model outperforms the state-of-the-an models in this experiment and similar to a model which is trained offline and after observing all the changes. The XdG outperforms all the other methods, but needs be informed of the task identity during the testing state.
We also visualise the forgetting matrix show how much the model forget the previously learned tasks, shown in Figure 16 In the forgetting matrix, the value at ieh column and jth row is the forgetting rate of task after learning task j. The proposed model does not forget too much learned tasks compared to other models. Notice that the forgetting rate of the second task is slightly higher than the first task the proposed model. We analyse this scenario as the different complexity of each task has impact on the performance of continual learning methods. Similarly, Lange et. al demonstrate that the order of the tasks can influence the performance.
4.2 SPI.TT CIFARI 00 EXPERIMENT
In the above experiments, we only test the model to continual learn five different tasks. In this experiment, we continue test the model in more difficult scenario. We split the CIFARI00 datasets into 20 different tasks, each of which contains five consecutive classes. The models have to perform 100-way classification task. Overall, the complexity of each task and the number of tasks to be learned are significantly increased compared with the split MNIST experiment. Fach expert contains three convolutional layers, each of which is followed by a max pooing layer. Then we add three fully-connected layer do the classification, and probabilistic layer containing five kernels to produce the task likelihood. We use the Adam optimizer with a learning rate 0.001.
We have shown the result in the Figure 2a The proposed model performs relatively well in this complex experiment and show that the proposed method can avoid the interference between the experts in the model. The XdG outperforms other methods, but it needs the task information during the test phase. We visualise the forgetting rate the Figure 2b) which shows that the proposed model forget less of the previously learned tasks than others.
5

---

Under review as a conference paper at ICLR 2021
TCNN
BI-R
0.8
0.5
Average Test Aopuracy
0.8
0.4
0.6
0.3
0.4
0,2
00.2
0.6
Join
2
0.0
-0.0
0.4
GR
None
0.75
0.60
0.8
BIR
0.45
0.6
0.4
10
Number of Tasks
(a)
(b)
Figure 2: (a). Average accuracy on the split CIFARI00 experiment. All the approaches are tested
in Class-IL scenario except XdG which perfroms Task-IL scenario. (b). Forgetting matrix of the
models in the CIFARI00 experiments. In the forgetting matrix, the value in column and jth row is the forgetting rate of task after learning task
Method Expert Gate
Split-
Split- CIII-CIFARROO
94.25
11.80
CN-DPM
93.81
20.10
TCNN
96.50
29.10
Table 2: Experimental results compared to the methods based on MoE.
For comparison, we report the accuracies of the Expert Gate, CN-DPM and TCNN the Table 2 TCNN outperforms the Expert Gate and CN-DPM without generative models.
5 DISCUSSION
In this section, take the split MNIST experiment example to analyse the overall performance of TCNN. We report the performance of TCNN under unknown task settings and visualise the density approximated by the probabilistie layer. Then analyse the limitation of the model and provide some possible solutions and preliminary results.
5.1 REVISITING THE LEARNED TASKS AND LEARN NEW TASKS
We analyse TCNN under the task unknown setup. The task-unknown settings represent conditions in which we do not provide the task information to the model at any given time (including training phase). In other words, the model sequentially learns different tasks without being informed of the task boundaries in the training state. The model turns into test mode after it is converged on the current task, and turns into training mode if it unconfident with the encountering task. We assume the tasks are coming in sequence. To further analyse the performance of the model, we test the model on the previously learned tasks after learning a new one. The tasks are coming in the following manner: the training set of Task 1, the test set of Task 1, the training set of Task 2, the test set of Task 1 and 2.... Note that the model may add expert unnecessarily if is unconfident with learned tasks, or fail to create expert if it is confident with unlearned task.
To detect the changes, we set threshold which is the mean - std the task likelihood of the model on the current training samples. If the task likelihood on the test samples is less than the threshold, the change is detected and start to learn a new expert.
We also conduct ablation study to compare the model with and without augmentation. As shown in Figure while the model tested on the learned task (green blocks), the task likelihood remains on a higher-level. While task transpires, the task likelihood decreases significantly (red blocks). The

---

Under review as a conference paper at ICLR 2021
1.0
MMM
10
VDM
0.8
0.6
0.6
Task Test Accuracy
Test Accuracy
0.4
0.2
02
Tack
Task 2
Task 3
Task 4
Tack 5
Task
Task
Task
Task
Task 5
(a) With multi-views
(b) Without multi-views
Figure MNIST experiment with and without multi-views augmentation. TCNN learns five different tasks sequentially without having the task information during both training and testing state. The blocks at the bottom represents different state. Blue blocks: The model tested on the learned tasks. Green blocks: The model is learning the current task. Red blocks: The model is tested on the unseen tasks. The red dashed lines are task boundaries. The blue shadows represent the area of mean(thh) + std(tth), where thh is the highest task likelihood generated by the model for the given input. When a change is detected, TCNN adapts to the new task automatically without forgetting previous ones.
average test accuracy shows that TCNN detects the task changes and adapt to the new tasks quickly without forgetting the previously learned ones. Compared to the model with multi-view augmentation, the model without augmentation perform well on detecting new tasks in the task stream. However, the test accuracy is not satisfied. Overall, TCNN provides unique and novel feature by automatically detecting and adapting to new tasks in an efficient way.
5.2 ANALYSING THE ESTIMATED DENSITY
In this section, we continue use the split MNIST experiment visualise the density estimated by the probabilistic layer.
As we mentioned in section 3.1 the kernels in the probabilistic layer act anchors to distinguish the in-task and out-task samples during the test phase. The hidden layers can be viewed as a feature extractor map the inputs the corresponding kernels. the expert observes the input beforehand, the features extracted by the hidden layers should match the kernels in the probabilistic layer. In contrast, the extracted features cannot match the existing kernels in the probabilistic layer expert has not seen the input beforehand. To verify this, we take the first expert in the split MNIST experiment and visualise the mean of the sample distribution, which extracted by the hidden layers of the first expert, from the first and second tasks. To visualise the overall distribution, assume the kernels in the first expert and the features extracted the two classes follows bi-variate Gaussian distributions. The results are shown in Figure The distribution of the feature distributions extracted from the samples from task and the kernels from expert 1 are overlapped. In contrast, the distribution of the feature distribution extracted from the samples from task is significantly different from the kernels in expert 1. Based on the Equation [] the task likelihood will be high if the kernel can describe the extracted sample distributions perfectly, e.g. Figure 4a| Alternatively, the task likelihood will be decreased if the kernels cannot describe the extracted sample distribution, e.g. Figure 4b
5.3 LIMITATIONS AND POTENTIAL SOLUTIONS
Based on the MoE framework, the demand of neuron resources increase linearly with respect to the number of learned tasks. However, each expert in the TCNN can be shallow neural network and
7

---

Under review as a conference paper at ICLR 2021
0.25
Task distribution
0.25
Task 2 distribution
0.20
Export 1 kernel
0.20
Expert kornel
0.15
0.15
0.10
0.10
0.05
0.05
0.00
0.00
-0.05
-0.05
0.0
0.1
0.2
0.3
0.0
Z1 0.3
0.2
21
(a)
(b)
Figure 4: Difference between the extracted samples distribution and the kernels in the probabilistic layers. We use the hidden layers of the first expert to extract the features of the samples from task and respectively. The task likelihood high if the two distributions are overlapped (c.g. Figure where samples are from Task 1, kernels are from: expert 1). The task likelihood will be decreased if they are different (e.g. Figure where samples are from Task 2, kernels are from: expert 1).
therefore the number of parameters used can be reduced. shown in Figure Sa the number of parameters used in TCNN is competitive to other methods after adding 20 experts.
Furthermore, Rusu al. 2016); Yoon al. (017)) re-use the neurons which have been trained with previous tasks. TCNN can be integrated with these techniques to save the resource. For example, we weighted sum the output features extracted by the convolutional layers of the previously learned
expert, and add it to the current extracted features progressively (Rusu et al. (2016)). Shown in Equation where [(()) is the feature extracted by the in expert, is number of tasks, ox trainable
weight parameter for 4z expert, fr()) is the final extracted features of expert T' Besides of that, we can also share the parameters of the learned experts the new experts directly. In this way, we do need to train the convolutional neural networks again but train weighted vector a instead, shown in Equation 8 We show the results in the Figure (5b] where the TCNN progressive train the expert progressively by Equation 7 the TCNN share is we train the last expert by the Equation [ After combining with the techniques to reuse the exisiting neuron resources, the performance of the models do not degrade and similar to the baseline TCNN. The results show that the TCNN can integrate with these techniques to use the neuron resources efficiently and reduce the number of parameters for the future experts.
fr(x)= = fr(x) + Σ
a4 f((x)
1=1
T-1 fr(x) = Σ a4 fz(z)
(8)
Besides the parameter explosion, the inference time of TCNN increases with respect to the number of tasks. During the inference phase, we have to iterate all the learned experts to give the final prediction. One possible solution for this is early existing paradigm. Instead of running the experts in parallel, put them in sequential manner. An expert will give prediction if its confidence above threshold, or it passes the data to the next expert. We leave the improvement TCNN and more comprehensive experiments future work.
8

---

Under review as a conference paper at ICLR 2021
20.0
1.00
17.5
Average Test Aocuracy
Paaammlers
15.0
0.99
12.5
0.98
10.0
0.97
5.0
TONN progressive
0.96
TONN
0.0
GR
TONN
ON-OPM
LwF
BI-R
2
Model
Number of Tasks
(a)
(b)
Figure 5: (a). Number of parameters used in CIFARI00 (b). Supplementary split mnist experiments for reducing complexity of the TCNN. The TCNN proggesssseeeveeee and TCNN share are trained by the Equation 7 and respectively. Overall, by using progressive training and parameter sharing techniques, the performance of the model does not degrade. The accuracy fluctuates around the baseline TCNN model.
CONCLUSION
In this paper, we first discuss the reasons for forgetting problem in machine learning when different tasks are given to model at different times. We demonstrate how providing acquiring the learning task information is essential to address this issue. We also present challenge that we have in our healthcare monitoring research and discuss how an automated and scalable model can solve this issue in dynamic and changing environments. We then propose Task Conditional Neural Network (TCNN) model for continual learning sequential tasks. TCNN is novel expandable model that utilises probabilistic layer to estimate the task likelihood.
TCNN consists of several different experts corresponding to different tasks. The model can learn and decide which expert should be chosen and activated under different tasks that are given to the model, without having provided the task information in advance. TCNN can detect the changes in the tasks and learn tasks automatically. The proposed model implements these features by using a probabilistic layer and measuring the task likelihood given a set of augmented samples associated with specific task. Our proposed model outperforms the state-of-the-an methods in terms of accuracy and the ability to learn the new tasks automatically. We hope this work can inspire more studies on investigating continual learning methods under dynamic environments. The future work will focus on transferring the knowledge between the experts, improving the task identification process, and defining dynamic and adaptive activation functions.
REFERENCES
Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert gate: Lifelong learning with a network of experts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3366-3375, 2017.
2017.
Wael Asaad, Gregor Rainer, and Earl K Miller. Task-specific neural activity in the primate prefrontal cortex. Journal of Neurophysiology. 84(1):451-45 1599, 2000.
Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-16077. PMLR, 2020.
Simon Haykin. Neural networks: comprehensive foundation. Prentice Hall PTR, 1994.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv preprint
2015.

---

Under review as a conference paper at ICLR 2021
Eiji Hoshi, Keisetsu Shima, and Jun Tanji. Task-dependent selectivity of movement-related neuronal activity in the primate prefrontal cortex. Journal of neurophysiology. 80(6):3392-339) 1998.
Nitin Kamra, Umang Gupta, and Yan Liu. Deep generative dual memory network for continual learning. arXiv preprint arXiv: 1710::1110.1030000 2017.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska. et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114 (13):3521-3526,, 2017.
Sang-Woo, Lee. Jin-Hwa Kim, Jachyun Jun, Jung-Woo Ha, and Byoung- Tak Zhang, Overcoming catastrophic forgetting by incremental moment matching. In Advances in Neural Information Processing Systems, pp. 4652-46566600 4662, 2017.
Soochan Lee, Junsoo Ha, Dongsu Zhang, and Gunhee Kim. A neural dirichlet process mixture model for fask-free continual learning. arXiv preprint arXiv:2001.00M0 .00689, 2020.
Shane Legg and Marcus Hutter. Universal intelligence: definition of machine intelligence. Minds and machines, 19(4):3914444, 2007.
HongLin Li, Payam Barnaghi, Shirin Enshaeifar, and Frieder Ganz. Continual learning using bayesian neural networks, 2019.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935-29M) 2018. 2018.
Saeed Masoudnia and Reza Ebrahimpour. Mixture of experts: a literature survey. Artificial Intelli- gence Review, 42(2):275-29) 293, 2014.
Nicolas Y Masse, Gregory D Grant, and David Freedman. Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization. Proceedings of the National Academy of Sciences, (15447))E1046771104777 2018.
Michael McCloskey and Neal Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Kuntal Kumar Pal and KS Sudeep. Preprocessing for image classification by convolutional neural networks. 2016 IEEE International Conference on Recent Trends in Electronics, Information & Communication Technology (RTEICT), pp. 1778-1781. IEEE, 2016.
German Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 2019.
Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2):
123-146, 1995.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint 1rXiv:1606.04671, 2016.
Joan Serrà, Dídac Surís. Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. arXiv preprint
2018.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems, pp. 2990-2999, 2017.
Donald F Specht. Probabilistic neural networks. Neural networks. 3(1): 1990.
Sebastian Thrun and Tom M Mitchell. Lifelong robot learning. Robotics and autonomous systems, 15(1-2):25-44 1995.
Gido M van de Ven and Andreas Tolias. Three scenarios for continual learning, arXiv preprint 1rXivv/904.07770 2019.
10

---

Under review as a conference paper at ICLR 2021
Gido M van de Ven, Hava T Siegelmann, and Andreas S Tolias. Brain-inspired replay for continual learning with artificial neural networks. Nature communications, 11(1):1-14, 2020.
Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv: 170.00017877 2017.
Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continuous learning of context-dependent processing in neural networks. arXiv preprint arXiv: /8100015/ 2018.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 3987- 3995. JMLR. org, 2017.


Reviews:
[{"review_id": "pTDveUHwy80", "reviewer": "Reviewer_NcCF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "It does make sense to try to use the MoE approach to handle continual learning (CL). The idea of learning task assignment probability is also reasonable to handle the task-agnostic CL problem. However, there are some problems with other motivations and approaches.\n1. Although the proposed method proposes to learn task-specific kernel/anchor to predict the task for a sample, it is unclear how the model will be able to identify the new tasks (i.e., varying of the tasks) during training. If the data from different tasks come in the data stream without known boundaries, I do not see obvious evidence and design in the model to learn the boundaries of the tasks. \nIf the exact task number K has to be given, it will be a very strong restriction for a model to handle the task-agnostic CL setting.\n2. It is not clear whether the model can explicitly identify the task boundaries via learning the task probability. If not, the approaches in (5) seem an ensemble of the MoE models training with different augmentation, and the task probability prediction works as the “heuristic attention”, which limits the novelty and significance of the work.\n3. Experiments on larger datasets (at least like tiny Imagenet) should be conducted. \n\nIf I understand correctly, the code link on page 4 should not be included. \n", "summary_of_the_paper": "This paper studies the task-agnostic continual learning problem by introducing a model called task conditional neural networks. The proposed method relies on the mixture of expert (MoE) approach to handle dynamically varying tasks and introduces “probabilistic layers” to predict the task index/assignment probability for each sample. ", "summary_of_the_review": "The basic ideas are well motivated, reasonable, and interesting. However, the motivation about how to approach the objective is improper, and the method is not carefully designed (or not completed). Experiments and analysis can be further improved. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "QUvRDbNGif2", "reviewer": "Reviewer_YEsG", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\nThe main idea is easy to understand. Task-incremental learning has been proven a much easier problem than class-incremental learning. Thus, a mechanism to predict the task identity will simplify the problem a lot. \n\nWeakness: \n1. It would be good to have a visualization to illustrate the high-level idea of the proposed method.\n2. The novelty of the method is limited. The idea of task predictor has been shown in [1].\n3. Catastrophic forgetting has always been a main issue in continual learning literatures. However, in this paper, how the proposed method contributes to alleviate forgetting is not clear. Moreover, is there a replay buffer to save previous examples?\n4. The motivation of (1)(2) is somehow weak. Why a MoE-like formulation is good for modeling task likelihood? We can have a MLP with a softmax activation to model task likelihood as well. For example, a task classifier in [1] works also quite well.\n5. For the experiments, it is not clear of how to set the augmentation functions and the parameters of the architecture is missing.\n6. Also, an ablation study will be very helpful to understand the method. For example, changing the number of experts per task, changing augmentation functions, etc.\n\n[1] Abati et al. Conditional Channel Gated Networks for Task-Aware Continual Learning. CVPR 2020.", "summary_of_the_paper": "This paper attempts to proposed a new method called Task Continual Neural Network to address the problem of task identity inference in continual learning. The proposed method estimates the task likelihood by constructing a probabilistic layer based on the idea of mixture of experts (MoE).  Experiments on benchmark datasets demonstrate the effectiveness of this method.\n\n", "summary_of_the_review": "Overall, I think the novelty of the method is limited, and critical insights and analysis are not enough, both theoretically and experimentally.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "PeDPZBF6t12", "reviewer": "Reviewer_sgG4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros: \n1. The paper is well-organized, easy to follow.\n2. The proposed method takes advantage of Mixture of Experts (MoE), which can use a shallower network for each expert. This saves memory and computational cost. \n3. The proposed method learns task-specific kernels to remember their data patterns for inferring the task identity, which is a new idea.\n\nCons:\nIt is difficult to infer task identity without directly accessing the old data distributions, and the proposed idea of learning task-specific kernels might be an effective solution. My concern is mainly on the empirical side:\n1. The evaluation is on small-scale images, i.e. split-MNIST and split-CIFAR100. The evaluation of more complex domains with a larger scale is necessary since it will be more difficult to learn the data patterns and infer the task identities.\n2. Replay-based methods are generally more advantageous in class-incremental scenarios since they directly recover old data distributions. However, the authors mainly compare with generative replay approaches. It would be helpful to compare with episodic replay approaches, such as iCaRL (CVPR17) and RWalk-S (ECCV18), and feature replay, such as REMIND (ECCV20) and Adam-NSCL (CVPR21).\n3. The authors claim memory and computation efficiency as a key advantage of their method. However, since the entire model is expanding in continual learning, a comparison of memory and computational cost with other baselines should be provided.\n\nPost rebuttal:\nAfter reading the reply, the author has done partial experiments that show the effectiveness of the method. However, there still needs further experiments that may require a new review. Thus, I keep my score.", "summary_of_the_paper": "This paper provides a new method to infer the task identity without directly accessing the old data distributions. The proposed method can learn task-specific experts with task-specific kernels to decide which expert should be chosen and activated under different tasks given to the model. The proposed method achieves strong performance on split-MNIST and split-CIFAR100 without requiring task information in advance. ", "summary_of_the_review": "Applying the MOE to continual learning is a new idea, but it should prove that inferring the task-specific identity has noticeable advantages over other memory buffer-based methods that are listed in the previous section, at least from the numerical perspective.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "4P_mO9yW-oV", "reviewer": "Reviewer_H38t", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The experiments do not report any indication of the size of the models for the different methods (e.g., number of parameters). The proposed method requires at least one expert per task and is therefore computational inefficient compared with other baseline.\n\nOn the subject of baselines, the baselines reported in this work are weak. It is unclear to me why some baselines were not reported despite being cited in the paper. For example,\n- Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016.\n- Joan Serrà, Dídac Surís, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. arXiv preprint arXiv:1801.01423, 2018.\n\nThe results reported (in Table 2) are not competitive with the state-of-the-art by some margin.", "summary_of_the_paper": "This paper introduces Task Conditional Neural Network for continual learning. The method is based on mixture of experts. Each expert is independent of other experts and therefore the model is not susceptible to catastrophic forgetting.", "summary_of_the_review": "Nice idea but weak experimental baselines and results.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an expansion strategy for both task agnostic and task-boundary aware CL. The authors demonstrate the quality of their method using two-standard scenarios with the Split-MNIST and CIFAR datasets. 

Enabling CL for task-agnostic and task-boundary aware is important and an active area of research. The proposed approach is an interesting method that adds an expert for each new task. Experts are then combined (Mixture of Experts) for prediction. One disadvantage of a MoE approach is that the model size and compute will grow linearly with the number of tasks. This effect is partly limited in the paper as the authors show that experts can be small neural networks.

There was a bit of confusion in the original reviews regarding the exact setting this paper works under. As far as I understand this paper mostly deals with the class-incremental setting (task IDs available at training time, but not at test time). The task agnostic setting (task IDs never given) is also explored in Section 5.1. I think this confusion is partly a reflection of the state of the CL literature and the authors provided clear and concise replies to the reviewers.

The main limitation that remains is regarding the experiments. I agree with the reviewers that the current experiments seem somewhat preliminary and showing results on larger scale datasets and/or compared to a wider diversity of baselines is important. Reviewer sgG4 made precise comments about this. Other minor comments by the reviewers including providing a detailed report of the memory usage and computational costs of the various methods (partly done in Figure 5.3).

I think this method is interesting and could be impactful. I strongly encourage the authors to polish their manuscript and consider adding some of the additional empirical results that were suggested.
Decision: Reject