Enhancing Knowledge Transfer for Task Incremental Learning with Data-free Subnetwork
Authors: Qiang Gao
Xiaojun Shan
Yuchen Zhang
Fan Zhou
Venue: NeurIPS 2023 poster (2023)
Keywords: data-free subnetwork
task-incremental learning
knowledge transfer
mask
TL;DR: We introduce a novel neuron-wise task incremental learning method, namely Data-free Subnetworks (DSN), which attempts to enhance the elastic knowledge transfer across the tasks that sequentially arrive.

Abstract: As there exist competitive subnetworks within a dense network in concert with Lottery Ticket Hypothesis, we introduce a novel neuron-wise task incremental learning method, namely Data-free Subnetworks (DSN), which attempts to enhance the elastic knowledge transfer across the tasks that sequentially arrive. Specifically, DSN primarily seeks to transfer knowledge to the new coming task from the learned tasks by selecting the affiliated weights of a small set of neurons to be activated, including the reused neurons from prior tasks via neuron-wise masks. And it also transfers possibly valuable knowledge to the earlier tasks via data-free replay. Especially, DSN inherently relieves the catastrophic forgetting and the unavailability of past data or possible privacy concerns. The comprehensive experiments conducted on four benchmark datasets demonstrate the effectiveness of the proposed DSN in the context of task-incremental learning by comparing it to several state-of-the-art baselines. In particular, DSN enables the knowledge transfer to the earlier tasks, which is often overlooked by prior efforts.

Enhancing Knowledge Transfer for Task Incremental Learning with Data-free Subnetwork
Qiang Gao!.,, Xiaojun Shan?**, Yuchen Zhang", Fan Zhou
Seuthweeserrn University of Finance and Economics, Chengdu, China University of Electronic Science and Technology of China, Chengdu, China qianggao @ swufe.ddu.cn, (xiaojunshan @ std., yuchenzhang @ std., fan.zhou @ Juestc.edu.cn
Abstract
As there exist competitive subnetworks within a dense network in concert with Lottery Ticket Hypothesis, we introduce novel neuron-wiss -wise task incremental learning method, namely Data-free Subnelworks (DSN), which attempts enhance the elastic knowledge transfer across the tasks that sequentially arrive. Specifically, DSN primarily seeks to transfer knowledge to the new coming task from the learned tasks by selecting the affiliated weights of small set of neurons to be activated, including the reused neurons from prior tasks via neuron- wise masks. And it also transfers possibly valuable knowledge to the earlier tasks via data- free replay. Especially, DSN inherently relieves the catastrophic forgetting and the unavailability of past data or possible privacy concerns. The comprehensive experiments conducted on four benchmark datasets demonstrate the effectiveness of the proposed DSN in the context of iisk-incremental learning by comparing it to several state-of-the art baselines. In particular, DSN enables the knowledge transfer to the earlier tasks, which is often overlooked by prior efforts.
Introduction
Continual learning (CL), one of the human-like lifelong learning incremental learning paradigms, has received enormous attention in Artificial Intelligence (Al) community due to its capability incrementally learn sequence of tasks in a (deep) neural network and to keep accumulating knowledge throughout its lifetime [. According to the context of whether the task identity is provided during model training and inference, the majority of prior works categorize continual learning into three main practical problems by specific settings, including task- mask-ineremental learning, class-incremental learning, and task-free incremental learning. This study was prepared for Task Incremental Learning or TII. for short. Not limited to TH., existing solutions for incremental learning are put forward to solve the catastrophic forgetting (CF) issue 22 that common phenomenon in CL scenarios. Usually, these approaches can be broadly categorized into three directions, i.e.,
regularization-basee
5 rehearsal-basee
66. g and architecture-based
I112.
Recently, several studies have demonstrated that deep neural networks are usually over peeettaaaetti, whereby the redundant or unuseful weights be pruned, allowing efficient computation and on-par or even better performance [I3 15). Likewise, researchers stated this phenomenon as Lottery Ticket Hypothesis (LTH) [I6 177, i.c., a randandy-inittalizzedd initialized neural network contains subnetwork such that, when trained in isolation, can match the performance of the original network. With this in mind, more recent efforts turn randomly initialize large but sparse neural network and employ it to sequentially discover compact subnetworks for (task) incremental learning,
Motivations:
"qqull contribution. Coorespponding author www..nusoueeeecccomm edu. cn).
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

aiming at providing room to learn new tasks as well as enabling forward knowledge transfer from the prior tasks [8] 19 20). Specifically, they concentrate on weight-wise search using an adaptive binary mask that determines which weight should be activated (winning tickets). Nevertheless, we consider that it is not preferable to maintain such a binary mask for retrieving a subnetwork, especially in incremental learning. Since it will destroy the continuous structure of layer-dependent subnetwork. Meanwhile, maintaining a binary mask whose size equals the number of parameters also results in significant resource consumption. In addition to this, knowledge transfer across different tasks is the core work of incremental learning. Existing solutions primarily seek to enable the knowledge transfer from earlier tasks to the new coming task forward knowledge transfer), along with considering the CF problem [2] [2 23]. Like human cognitive processes [24]. we debate whether the newly learned knowledge has the potential to help improve past tasks (i.e., backward knowledge transfer). Intuitively, it is natural and practical that we can replay the old samples from earlier tasks, like theersal--bssdd approaches, trigger the ability tackle old tasks much better. However, this is not applicable in the actual incremental learning scenario due to computational overhead and privacy concerns. With those in mind, we ask much more ambitious question: Does LTTH hold in the setting of TIL concerning elastic knowledge transfer when different tasks arrive sequentially?
Contributions:
To answer this question, we propose novel and practical approach named Data-free Subnetworks (DSN), enabling the elastic knowledge transfer across the tasks that sequentially arrive. Motivated by LTH, we assume that there exists hypernetwork (Fig. (a)) that contains a sequence of competitive "ticket" nub-networks, where each of them can perform well on the affiliated task with the knowledge transfer in mind. DSN mainly contains two components. i..., neuron- wise mask and data-free memory reply. Specifically, we first randomly initialize a hypernetwork and incrementally learn the model parameters and task-specific masks, where each mask determines which neurons and their corresponding weights should be used for a new coming task (Fig. [ (b)). To allow the forward knowledge transfer on a new task, we use the mask to adaptively select the neurons that have been used in the earlier tasks and make their corresponding weights unchangeable for the purpose of addressing the CF problem. To permit the backward knowledge transfer, we first measure the mask similarity scores and craft the impressions of the most similar task via data-free memory replay. In contrast sample replay paradigms, we treat the subnetwork as a past experience and use to recall past impressions, avoiding incrementally preserving the past actual samples. Then, we fine-tune the most similar task and make backward knowledge transfer possible(Fig. (c)). We summarize our contributions as follows:
In concert with Lottery Ticket Hypothesis (LTH), we introduce a novel and practical task- incremental learning solution DSN, which aims to learn compact subnetwork (winning ticket) for each task from the perspective of knowledge transfer, including forward and backward transfer.
We devise a neuron-wise mask mechanism to adaptive select neuron-affiliate weights to transfer the learned knowledge the new task, where the used neuron-affiliated weights in the past tasks are frozen to eliminate the CF problem. Besides, our proposed data-free replay mechanism regards the trained subnetwork as a past experience and uses craft impressions regarding the past samples, which does not require holding any actual samples related to past tasks.
The comprehensive experiments conducted on four benchmark datasets demonstrate the effective- ness of our proposed DSN against the sfate-of-the-art baselines.
2 Related Work
Catastrophic Forgetting, Existing approaches to address the CF problem can be broadly categorized into three directions. (1) Regularization-bann based approaches that add a selective penalty in the loss function, which penalizes the variation of network parameters depending on its importance in
performing previous tasks 3 25 26 27.. EWC operates the Fisher information matrix to estimate the importance weights 3. Synaptic Intelligence (SI) provides an online approximation of parameter importance by assessing their contributions to the variations of total loss (28. (2) Rehearsal- bassdd approaches typically incrementally hold a few representative old training samples as a small memory
buffer and replay to retain the past knowledge when learning a new task 6 7 R 29
30 31).
For example, GEM proposes to further guarantee an equal number of old training samples per class [32] DGR presents a framework that replays generated data sampled from the old generative model while learning each generation task to inherit the previous knowledge (6). Also, other

---

(8) Dense Intiallzztion
(b) Selection for Task
Consolidation for Task
Figure I: Task Incremental Learning with DSN: (a) We define the task network as a hypernetwork that can maintain the training of multiple tasks. And it will be randomly initialized (blue line) for task 1 training (pink circle and line); (b) We use the Neuron-wiss Mask produce a subnetwork for task t training (purple circle and line); (c) If task is the most similar to task by similarity measurement of masks, update the architecture of task based on the performance of Data-free Replay.
perform data-free generative replay to alleviate memory space or privacy concerns. For instance, [44] employs two teacher models to enhance the quality of generated samples. However, it requires an additional generative model, yielding additional trainable parameters. [35 can generate old samples merely through the main network. Besides, [33 35), which directly employ synthesized samples to fine-tune the model, have limitations in achieving forget-free performance and are susceptible to interference from other tasks. (3) Architecture-bat -aase approaches assign different model parameters to different tasks, freeze the parameters for old tasks by masking them out when training the new task, and expand the network when necessary [10 11
researchers K
35]
8 [9]. DEN
36) can dynamically adjust its network capacity as it trains on a sequence of tasks to learn a compact overlapping knowledge-sharing structure among tasks. WSN aims to explore the subnetworks sequentially for sequence of tasks from a sparse neural network [19]. It also maintains binary mask whose size equals the number of parameters, resulting in significant resource consumption. We argue the architecture-base approaches provide us with the possibility to explore better subnetworks while they either confront the redundant neurons (weights) or shallow knowledge transfer.
Knowledge Transfer. Recently, researchers and practitioners have argued that the ability to learn from sequence of tasks is in essence to complement and reinforce each other through knowledge transfer. Several studies have proposed improve task performance by maximizing shareable knowledge [37
[38 40).
For instance, Bayes model [1 and regression method [43 concentrate transferring
the learned knowledge to facilitate learning in the new domains or new tasks. BNS [12 presents
Task-CL. algorithm that can adaptively determine the network structure for each new task and selectively transfer previously learned knowledge. However, it employs massive past samples to address the negative knowledge transfer concern. HAT [44] and WSN [19] selectively reuse the optimized parameters via a binary masking mechanism, enabling significant forward knowledge few-shot scene, which is accomplished by jointly learning model weights and adaptive non-binary soft masks, with the major subnetwork minimizing catastrophic forgetting and the minor subnetwork preventing overfitting. Also, fails to take backward knowledge transfer into consideration. [46] proposes two distillation-bu bbased objectives for class incremental learning. This method can utilize the feature space structure of the previous model to preserve the representation of previous classes. Besides, a controlled transfer is introduced to approximate and condition the current model on the semantic similarities between incoming and prior classes, maximizing positive forward transfer and minimizing negative backward transfer. However, it can not achieve a positive backward transfer. [38] and preserve previous data as a replay buffer for retraining old tasks, which raises concerns regarding explicitly identify and transfer from previous tasks to new ones, designed specifically for LLMs. [48
transfer. However, backward knowledge transfer is not considered. [5) focuses
data privacy and memory overhead. 33 utilizes CL-plugin
useful features//nowledde characterizes task correlations, identifies positively correlated old tasks, and selectively modifies the learned model of the old tasks when learning new tasks. In sum, the above solutions concentrate on either addressing severe negative transfer using pruning methods or using additional data to help knowledge consolidation. In contrast, our DSN attempts to select subnetworks from a sparse network for a sequence of tasks while using data-free mechanism to enable the possibly positive knowledge transfer to the old tasks.
3

---

3 The Proposed Method
3.1 Problem Statement
Without loss of generality, classical TIL scenario, typically in supervised learning setting (c.g.. classification), deals with a D = (Di, D2. ........................ Dr} of tasks that come in turn the learner or task network (e.g., a deep neural network). Let D; { 3i) denote the dataset of task t, containing nu pairs of raw instances and independent label space. In our context, assume that we have a neural network, named hypernetwork H. Hi- (n)) is composed of layered neurons n where learnable weights bridge them together. TIL. follows the common continual learning paradigm and attempts to sequentially learn a set of task T with the following objective for each task t:
1 T1
0- minimize 0
C(H(a):0)vvl
(I)
where -(:,)) is loss function such as cross-entropy loss. It is important note that D, for task t is only accessible during task learning, and any instances in Dr cannot be accessed while learning future tasks. Furthermore, assume that task identity is given in both the training and inference processes in the TIL scenario. In addition, we follow most of the previous studies and employ multi-head settings for TIL, with each task having its own single but non-overlapping head.
The Lottery Ticket Hypothesis (LTH) hypothesizes the existence of competitive subnetworks in randomly initialized dense neural network [16], which motivates us to ingeniously devise an incremental learner that employs an over preammarmeterieeed deep neural network to allow more room for future task learning. Therefore, our goal is discover optimal subnetwork along with neuron-wisg wise mask m, E (0, 1, for each task t such that |mj] << [nl, summarized as follows:
1
Σ c (H (at O(n O m.) vi) - CHH(x)) (), 0), vi)
0", m = minimize
0,T,, Thp
s.l. mm << |m].
3.2 Neuron-wise Mask
To obtain an optimal subnetwork from the hypernetwork H. for any new coming task, we are motivated by HAT (4 and inhibitory synapse [49] and devise a neuron- wise differentiable mask mechanism to search a small set of neurons with the objective 22. Specifically, for each layer in the hypernetwork H. let each layer associated with a learnable embedding el (1 21, L}) before training a coming task f, where I is the number of layers. To determine if a neuron in should be used for task t, we adopt the binary mask m, to produce the architecture of the subnetwork. However, neurons typically have only two states, i.e., activated or not activated. To this end, we use the following function to bridge the gap between the layer embedding ei and layer mask m E m,
m = 0 e.),
where a is the sigmoid function. is the scaling factor to control the sharpness of the function that can make the binarized progressively [44 [5]] We note that the optimal mask m, can be received by jointly optimizing the model parameters θ of task f, as the task learning is conditioned on mask m, For instance, the forward propagation of hypernetwork H during the training of task can be expressed as follows:
forward: h h O m
where h denotes the output of neruons in the layer 1. Correspondingly, the backward propagation can be summarized follows,
ac
backward: 0uj Gug
O max(m Lit m m1-1.)),
(5)
00
where 0liy 0 refers to parameter (i.e., weight) between the neuron i of 1-th layer and the neuron
j of (1-1)-th layer. mi denotes the mask value of i-th neuron. max(m,)) 1. -m -1 indicates that the parameter will not be updated if neither the neuron i of I-th layer nor the neuron j of - 1-th

---

layer is activated for task t. Otherwise, the parameter will be updated. However, the above backward propagation inevitably results in the catastrophic forgetting issue since the weights trained in the previous tasks could be changed. Hence, we regard the used neurons in the previous tasks as the synapses that only take the role of sending messages between different layers. To this end, we devise a cumulative mask to maintain the signal that all previously learned parameters should be frozen. The cumulative mask mgt for any task t can be recursively obtained as follows,
mige = m m4)-
(6)
Note that we set mcc O, indicating all neurons can be used before training task 1. Thus, the Eq.(5) can be revised as follows.
dc 00uj
backward: Ong Ons
@ max(m))
@ (1-m (t) my 14
(1 m (2)
worth noting that mask operation may also pose capacity problems. To this end, we add a regularization term L, to the objective L to hold more room for future tasks as follows,
C. = E m (1 11-mzz
(8)
E m2)
where nu denotes the neuron number of (-th layer in hypernetwork H and 17 is a hyperparameter that controls the capacity preference for the current task Note that we provide more details of the hyperparameter setting in Appendix A.
3.3 Data-free Replay
To enhance the knowledge transfer from the current task to the earlier tasks, data-free replay recalls network experiences to address the unavailability of old samples or possible data privacy concern.. Recent zero-shot learning solutions (5] provide us with a new perspective. That is, we craft past knowledge through the subnetwork rather than seeking past samples.
Output Space Modeling. Let Og denote the output space (i.e., Softmax space for classification problems) of task i over the subnetwork H(- O(n m.)). Recent studies (51 reveal that any output representation can be sampled from a Dirichlet distribution as its ingredients fall the range of [0.1] and their sum is 1. Specifically, the distribution to represent the (Softmax) outputs of of c-th class can be modeled as Dir(C,, B x a)), where e E {1, 2, .... C. is the class index regarding task a E RC, is the concentrate vector model c-th class, 3 is scaling parameter [2] [T), and any real value in a is greater than O. It should be noted that there could exist interactive correlations between different classes in a single task since the samples related to different classes may be similar. To mitigate the risk of ignoring the inherent distribution of different classes of samples, we are inspired by [51 and additionally preserve a class similarity matrix M, after training each task to describe the correlation between different classes. As M, of task t is class-wir table, it will not bring the significant capacity issue. We detail the motivation of the similarity matrix Appendix
Replay with Impression Craft. Due to the unavailability of old task samples, like the human memory mechanism, we attempt to synthesize the input of H(, O(n O m))). which can be denoted as an impression regarding a raw sample in an earlier task. To be specific, we synthesize an impression set I, based on the Dirichlet sampling. For any synthesized sample 2 corresponding to c-th class in task f, we initialize af* to random noise, e.g., by sampling from uniform distribution. And we optimize with the following objective over a sampled outputs of and the subnetwork H(-, O(n 3 m.)), i.e.
# arg 0(n @ m)),))) o5).
where 7 is a temperature value 533 used in the output (Softmax) head. In this manner, we can successively produce impressions for each class. Intuitively, we can produce the impressions of different classes equally. However, the subnetwork may show different performances in different classes. That is, we need to consider the emergence of hard classes. Thus, we generate biased number of impressions of different classes based on the accuracy performance. For each task learning, report the error rate of each class and normalize them as the distribution of the sampling rate.
5

---

Algorithm 1: Incremental learning with DSN.
Input: (D:Y=1. hypernetwork H. with neurons n and associated parameters 0, cumulative mask mgg.. Randomly initialize 0 and set m. = 0:
for t = 1, .., T do
Initialize layer embeddings e, = {e))
Use el f= to obtain the mask ma according to Eq
Train the hypernetwork with (n O m. on Dr according to Eq. and Eq. 8):
Obtain the optimal layer embeddings e, and layer masks mi:
Update the cumulative mask m via 6).
if t>> then
Mcasure the task similarity scores St via Eq. Obtain the most similar task argmax with number of class (S))
Compute the replay number of cach class {Bih (xzanttt)) according to the class spes-iic error rate; Initialize impression set (s)) <- {): for 1 Carmmn(())) do Set the concentration parameter o" M (8))
for b = B1, B2. ... BC do
for i= :1:bb do Sample (S) ~ Dir Dir(Capma((S)) Sy x o²); Initialize 2 (Sgmax(S)) to random noise and craft (S,) via (9. Iugmax(S.)) (S, fc "Sg)
Merge mask for task argmax((Si): ((rggaa((S))) = max (m M (S)) m,); Fine-tune tune the task argmax(S.) based on Iangmas(s.) S.) and (Pa)) if accuracy performance of task (rgmax(St) is higher than before then update the mask m (S)
Output: Optimal hypernetwork with parameters 0" and task specific masks (m; )T.
3.4 Knowledge Transfer-enhaneed Incremental Learning with DSN
Incremental Learning (L 2: 1). For the first task, i.e., / 1, we randomly initialize the parameters 0 the hypernetwork H. For each task / including the first task, we start with randomly initializing task-specific classifier (e.g., fully-connected layer with Softmax function) and the corresponding layer embeddings et. Then, we attempt to jointly learn the model parameters in the hypernetwork H. and task-specific masks of subnetworks regarding each task, where each task-specific mask determines which neurons will be used in the current task. Specifically, given a task f, we optimize the parameter and its mask m. with the objectives of 2q and (8) where the update of parameters will follow the rule of Eq. 7 for the purpose of addressing the CF problelmm. After the training, we likewise build the class similarity matrix M, and update the cumulative mask m.r..
Knowledge Transfer > 1). To consolidate the possibly useful knowledge behind the newly learned task to the earlier tasks, we first measure the task similarity and then employ our date-free replay to transfer the knowledge to the most similar task. Notably, we could choose multiple similar tasks to transfer, but this would take more transferring time than it is worth due to the cost of memory replay. Since the unavailability of any real samples in the earlier tasks, it is impossible to use the naive solution for task similarity measurement, i.e., estimating the similarity between the instance distributions of two tasks. Fortunately, we can use the masks tasks to measure the task similarities. For current task t,
compute the similarity scores St follows,
S. (cosine(m;, mo), cosine(m))) (m,, m.), cwsine(ms......c m. 1)};
(10)
where cosine is the cosine distance. As such, we can obtain the most similar task that owns the largest similarity score, i.e., argmax(S,). After that, we can use the data-free fata-free memory replay produce impression crafts as the input samples of the most similar task by optimizing the objective of Eq. noted that we only use the impressions to determine whether we need to adjust the subnetwork architecture of the most similar task. Meanwhile, any parameter update in the hypernetwork will cause interference with the current task as well as all the previous tasks. Instead, we only allow the parameter update in the task task-specific classifier (head) while freezing any parameters in the subnetwork.

---

Specifically, we first merge the optimal mask of current task t to the most similar task argmax(S (St), i... arrgmax(S)) max(m max "rgmax(S.)) m;). Then, we fine-tune the task argmax(S,) based on margmax(S)) and make knowledge transfer possible. We will update the architecture of the subnetwork regarding task argmax(S,) if it performs better. Notably, the cumulative mask mg does not need to change due to the fact of (argmax(S.) C mgg. The complete workflow is summarized in Algorithm |
4 Experiments
Datasets. We employ four benchmark datasets for TIL problem as follows: Permuted MNIST
(PMNIST) 33, Rotated MNIST (RMNIST) [21]. Incremental CIFAR-100 [54] [55] and Tiny
geNet 19. PMNIST encompasses 10 variations of MNIST [56), wherein each task is transformed
by fixed permutation of pixels. RMNIST also comprises ten versions of MNIST, with each task rotated by a specific angle between 0 and 360 degrees. The original CIFAR- 100 was divided into 20 tasks, each containing 5 different categories. IinylmageNet constitutes a variant of ImageNet [57]. we construct twenty 1 10-way classification tasks for consistency in our experiments.
Baselines. We compare our DSN with the following representative baselines: SGD 158 utilizes a feature extractor without any parameter update and learnable classifier solve series of tasks incrementally. EWC (3] purposefully uses Fisher information as a simple and effective standard baseline to alleviate CF. IMM [25) aims penalize parameter modifications, yielding two variants, i..., Mcan-IMM and Mode-IMIn PGN [9] attempts to expand the task network by incorporating a fixed number of neurons. DEN
36 dynamically decides the number of new neurons via selective retraining and network split. RCL 21 is a reinforcement learning method that controls the scale of newly added neurons while rendering weights relative to the used neurons unchangeable. HAT 4 leverages a masking mechanism to prevent the optimization of old neurons. SupSup [22] solves the CF problem when sequentially learning tasks by discovering super masks (sub-networks). WSN [I9 tries to learn the model parameters sequentially and select the optimal sub-network for each task.
Implementations and Evaluation Metrics. In accordance with the task-ineremental continual learning framework, all methods used in the experiments employ a multi-head configuration. For two variants of MNIST, we adopt the experimental setup outlined in [4 in which we use a two-layer MLP, a multi-head classifier, and begin with 2000-2000-18 10 neurons for the first task. For the other datasets, we refer to previous works 19 44 and use the modified model with three convolutional layers and two fully- fully-connected layers. We follow previous studies 12 19 and evaluate all methods on three metrics: ACC, BWT, and Trans. ACC is a common CL metric that reports the average accuracy of all tasks validated on their respective test sets after training on all tasks. BWT measures how the model performance changes for the old tasks once the model is trained on the new task. Trans measures the proficiency to transfer knowledge from previously learned tasks to newer ones, indicating the usefulness of the acquired knowledge in facilitating new learning tasks. Notice that more details of the implementation setup are specified in Appendix C. The source codes are available at whhtpss/// //githhb. com/
wwt./ooo.//.////// ssssaiiiie ao un/ Jn/DSSN
Table 1: Performance comparison of the proposed method and baselines four datasets.
Model
PMNIST
RMNIST
CIFAR-100
Tiny ImageeeeNtt
ALC(5)
[[S]
[rans[Se]
ACCTR) )
BW ((5)
Transt
ACL(S)
HW (T3)
Transt 90)
ACC(S)
T(S) Trans(Wy
SGD
81.37
-24.52
-17.06
72.83
-25.32
-25.08
59.82
-24.09
-24.02
30.24
-19.12
-19.9
EWC.
94.20
-0.32
-4.3
94.86
.0.73
-3.05
67.15
-8.61
-1.699
40.85
-5.24
-9.35
mean-IMM MMan-IMMM
80.10
-1.13
-18.33
88
-0.96
-9.10
56.08
0.23
-27.76
30.10
-3,21
-20.10
mode-IMM
93.13
-4.17
-5.30
89.48
-7.40
-8.43
61.22
-21.49
-22.62
32.26
-19.02
-17.94
PGN
91.89
0.00
-6.54
90.01
0.00
-7.90
53.84
-14.66
-30.00
24.47
-12.12
-25.73
DEN
91.96
-0.41
-6.47
91.53
-0.52
-6.38
59.32
-1.24
-12.79
33.86
-3.88
RCL
92.28
0.00
-6.15
93.97
0.00
-3.94
61.77
0.00
-22.07
38.23
0.00
-11.79
HAT
97.10
0.00
-1.33
97.49
0.00
-0.42
71.23
0.00
-12.61
44.51
0.00
-5.69
SupSup
97.02
0.00
-1.41
97.15
0.00
-0.73
71.44
0.00
-12.40
43.22
0.00
-6.98
WSN
97.16
0.00
-1.27
97.32
0.00
-0.59
72.84
0.00
-11.00
45.96
0.00
-4.24
DSN
98.24
0.01
-0.19
97.73
0.02
-0.18
75.17
0.02
-8.67
46.56
0.04
-3.64
Overall Performance. To examine the influence of task mixture, we shuffled the tasks with five different seeds, resulting in five lists with different task orders. The averaged results over five runs are presented in Table where the deviation results are reported in Appendix D. We have the following observations: First, not all CL solutions outperform the naive SGD method, even mean-IMM performs slightly worse PMNIST than SGD. We consider that this observation may be due to the sequential
7

---

Avergg A00%%)
Averge
Increasing No. Taaks
Increading No. Tasks
Increading No. of
(a) RMNIST
(b) CIFAR100
(c) TinylmageNet
Figure The accuracy performance during entire incremental learning.
weight penalty during knowledge transfer, which could significantly hinder knowledge transfer between old and new tasks. Second, we find that CL methods with network expandability such as PGN and RCL do not perform as well as EWC. This indicates that merely iterative network expansion, without network pruning or neuron selection, has the potential of incurring an over ver-parameterized risk [I3 59]. On the other hand, HAT, SupSup, and WSN adhere to the merit of network pruning and mainly seek to produce sub-network by partially selecting some neurons (weights), resulting in higher accuracy performance. However, they can only facilitate forward knowledge transfer, yielding only narrow knowledge consolidation. In contrast, our DSN outperforms all the baselines on all metrics such as achieving the 3.20% improvements on CIFAR- 100 regarding ACC. In addition, to DSN in incremental learning from the fine-grained aspect, Fig. presents averaged results as the number of tasks increases, we can clearly observe that DSN outperforms other baselines during the entire incremental learning process.
show the superiority
2
Forward Knowledge Transfer. The results of Trans in Table T demonstrate that DSN is substantially better overall due the flexible neuron selection. Specifically, DSN performed on PMNIST and RMNIST shows only slight degradation over the single-task learning scenario. This finding indicates that we can use more compact net-nesworks to take on multiple tasks incrementally, which is more practical in future computing contexts, such as edge computing and low resource constraints.
51
5
Accuracy DJ)
Tack
12 1811
10 17778
(a) RMNIST
(b) CIFARI00
(c) TinyImageNet
Figure 3: The accuracy performance of the first task in incremental learning.
The experimental results BWT indicate that SGD suffers from severe knowledge forgetting or negative knowledge transfer issues as does not take any effort to avoid alleviate the catastrophic forgetting problem task incremental learning. Other traditional CL approaches, such as EWC and LMM, choose to confine the shift of model parameters to maintain the old task's performance while learning new tasks sequentially. However, the penalty parameters still affects the ability of the model to recall old tasks, and these methods also confront severe catastrophic forgetting problems. In comparison with forget-free freeet-free methods like RCL, HAT, and WSN (BWT=0), DSN can even promote the previous task performance on four datasets (BWT>0). There are two reasons this observation. First, existing forgetting-fis free solutions concentrate on simply freezing the neurons or weights used in the previous tasks while making the architecture of the task network regarding each old task unchangeable throughout the entire incremental learning process. As result, it offers only barely forgetting no-forgetting gains. Second, our DSN owning data-free
Backward Knowledge Transfer:

---

replay enables the possibility of actual backward transfer, i.e., transferring new knowledge to previous tasks. Fig. shows the performance of the first task throughout the incremental learning. It obvious that HAT and WSN can only maintain the model performance on the first task while they cannot promote the performance of the first task when incrementally receiving new tasks. This suggests that newly acquired knowledge can help address inadequate experience or knowledge of past tasks.
Cank
Userl Ni wnw.rommaa
66-
Used
16-
20
30
(a) PMNIST
(b) RMNIST
(c) TinylmageNet
Figure 4: The layer-wise neuron usage in incremental learning.
Capacity issues. Fig. shows the capacity monitoring as the sequence of tasks emerges, where only show the first two fully-connected layers of TinyImageNet for better visualization. From macro perspective, we can find that the total usage (dashed line) of neurons increases incrementally more task comes. But discover that the used neurons for each task gradually increased, which suggests that DSN prefers to reuse more neurons from earlier tasks when a new task arrives. Especially, DSN reuses more neurons from the past on RMNIST as the total usage increases slowly. The rationale behind this is that although different tasks have different angles of images, they do change the goal of image classification and those that arrive in sequence are implicitly similar. Furthermore, we investigate the scale impact of our defined hypernetwork. Specifically, we vary the neuron number of each layer in the hypemetwork with different initial learning rates. As shown in Fig. it is evident to observe that a larger over-paree parameterized hypernetwork enables higher accuracy performance. Interestingly, we find that a smaller learning rate brings us higher gains. We conjecture this phenomenon indicates that over-fitting problem could degrade the model performance.
200
300
xi2 un, 1M..5 200 Loacting Rata
casl Fhn
(R arbuus Lanni Rate
150g
(a) RMNIST
(b) CIFAR-100
(c) Tiny InylmageNeet
Figure 5: Hypernetwork capacity in incremental learning varying different learning rates.
Efficiency Issue. We investigate the efficiency concern regarding our DSN and the representative baselines. First, Table 2 reports the runtime of all approaches including DSN. We first observe that solutions with network expandability take significantly more time, especially for RCL with reinforcement learning. The methods with parameter penalty spend less time while subnetwork selection methods such as WSN cost slightly higher. As for DSN, costs more time but less than network expandability-basee -based methods. There are two reasons: 1) recent mask mechanisms in recent subnetwork selection methods such as WSN are weight-wise where the constructed binary masks have the same shape the weights. As such, we couple with them in parallel manner. 2) DSN needs to craft the impressions regarding the old tasks, yielding higher time cost. Nevertheless, Table TableBshoww shows that DSN and HAT need less trainable masks than WSN due to the neuron- wise masks.

---

Sensitivity Analysis. We evaluate the impact of that aims to hold more room for future tasks. As illustrated Fig. 6 we can find that larger 7 will leave more room but bring slight performance drop. O indicates that we do not penalize the number of activated neurons for each task. We can find that the capacity full while the accuracy performance is the worst. We note that we evaluated the impacts of other key hyperparameters. However, due to space limitations, more details are described Appendix D.
Table 2: Statistics for full training and inference time. where 'h' represents hours.
Dutasel
SGD
EWC
Mean-IMM
mode-IMM
PGN
DEEN
RCL.
HAI
SupSup
WSN
DSN
PMNIST
1.24h
1.36h
1.4hh
1.38h
2.52h
41.54h
36.18h
1.58h
1.55h
1.65h
2.43h
RMNIST
1.17h
1.64h
188hh
1.0hh
2.12b
20.17h
14.0th
1.49h
1.32h
1.57h
2.18h
CIFAR-100
0.09h
0.11h
0.24h
0.12h
0.52h
9.08h
8.4h
0.13h
0.30h
0.37h
1.21h
TinyImugeNe
0.28h
0.32h
0.63h
1,07h
1.32h
23.45h
21.17h
0.35h
0.74h
0.81h
1.54h
Table 3: Total number of masks.
Dataset
HAT
SupSup
20.
P.MNIST
4.4K
40.0M
Acturanc
R-MNIST
4.4K
40.0M
CIFAR-100
93.4K
90.1M
Tiny Nntgenet
93.4K
90.I
140.75
Dataset
WSN
DSN
P-MNIST
Capecitty 80
Gapacityy (6)
R-MNIST
(a) PMNIST
(b) TinyImageNet
CIFAR-100
90.1M
93.4K
Figure 6: The impact accuracy and network capacity.
Tiny ImageNNt
90.1M
93.4K
5 Conclusions
Motivated by Lottery Ticket Hypothesis, introduced Data-free Subnetworks (DSN) approach for task incremental learning, aiming at enhancing knowledge transfer across sequentially arrived tasks. DSN consists of two components, i.c., neuron-win wise mask and data-free replay. The former aims to find an optimal subnetwork using task-specific masks for a new arriving task while the latter attempt craft the impressions for transferring the knowledge the past tasks. DSN allows for more flexible knowledge transfer between old and new tasks, whereas backward knowledge transfer was usually ignored before. The experimental results on four datasets show the superiority of our proposed DSN. As part of our future work, we plan to investigate how to devise a knowledge-inspired mask mechanism enhance knowledge transfer as more tasks arrive sequentially.
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China (Grant No.62102326 and No.62072077), the Natural Science Foundation of Sichuan Province (Grant No.2023NSFSCI411 and No. N0.2022NSFSC0505), and the Guanghua Talent Project
References
[1] M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions pattern analysis and machine intelligence, 44(07):336633385 2022.
[2] Michael McCloskey and Neal J Cohen. Catastrophic interference connectionist networks:
The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109-165. Elsevier, 1989.
[3] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Overcoming catastrophic forgetting neural networks. Proceedings of the national academy of sciences, 1 114(13):3521-332 2017.
et al.
10

---

[4] Sangwon Jung, Hongjoon Ahn, Sungmin Cha, and Taesup Moon. Continual learning with mode-importance based adaptive group sparse regularization. In Advances in Neural Information Processing Systems, volume 33, pages 3647-3658, 2020.
[5] Hao Liu and Huaping Liu. Continual learning with recursive gradient optimization. In Interna- lional Conference on Learning Representations, pages 1-18, 2021.
[6] Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. Proceedings of the 31 International Conference on Neural Information Processing Systems, pages 2994-3003, 2017.
[7] Lukasz Korycki and Bartosz Krawezyk. Class-incrementer experience replay for continual learning under concept drift. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3649-3658, 2021.
[8] Haiyan Yin, Ping Li, et al. Mitigating forgetting online continual learning with neuron calibration. In Advances in Neural Information Processing Systems, pages 10260-10272, 2021.
[9] Andrei Rusu, Neil Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv: 1606.0467,, 2016.
[10] Shuaicheng Niu, Jiaxiang Wu, Guanghui Xu, Yifan Zhang, Yong Guo, Peilin Zhao, Peng Wang, and Mingkui Tan. Adaxpert: Adapting neural architecture for growing data. In International Conference on Machine Learning, pages 8184 8194. PMLR, 2021.
[11] Yaoyao Liu, Bernt Schicle, and Qianru Sun. Rmm: Reinforced memory management for class-incremental learning. In Advances in Neural Information Processing Systems, volume 34, pages 777-33490 2021.
[12] Qi Qin, Wenpeng Hu, Han Peng, Dongyan Zhao, and Bing Liu. Bns: Building network structures dynamically for continual learning. In Advances in Neural Information Processing Systems, volume 34, pages 20608-20620, 2021.
[13] Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. In Advances in neural information processing systems, pages 1135-1143, 2015.
[14] Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. In International Conference on Learning Representations, 2017.
[15] Qiang Gao, Zhipeng Luo, Diego Klabjan, and Fengli Zhang. Efficient architecture search for continual learning. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[16] Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint TrXiv:/803.03635 2018.
[17] Eran Malach, Gilad Yehudai, Shai Shalev-Schwartz, and Ohad Shamir. Proving the lottery ticket hypothesis: Pruning is all you need. In International Conference Machine Learning. pages 6682-6691. PMLR, 2020.
[18] Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. Long live the lottery: The existence of winning tickets in lifelong learning. In International Conference on Learning Representations, 2021.
[19] Haeyong Kang, Rusty John Lloyd Mina, Sultan Rizky Hikmawan Madjid, Jachong Yoon, Mark Hasegawa-Johnson Sung Ju Hwang, and Chang D Yoo. Forget-free continual learning with winning subnetworks. In International Conference Machine Learning, pages 10734-10750. PMLR, 2022.
[20] Jian Jiang and Oya Celiktutan. Neural weight search for scalable task incremental learning. In Proceedings of the IEFF/CVF Winter Conference on Applications of Computer Vision, pages 1390- 13000-199,2000 2023,
11

---

[21] Ju Xu and Zhanxing Zhu. Reinforced continual learning. In Proceedings of the 32nd Interna- tional Conference on Neural Information Processing Systems, pages 907-916, 2018.
[22] Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Raste- gari, Jason Yosinski, and Ali Farhadi. Supermasks superposition. In Advances in Neural Information Processing Systems, volume 33, pages 15173- 15184, 2020.
[23] Pratik Mazumder, Pravendra Singh, and Piyush Rai. Few-shot lifelong learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 2337-2345, 2021.
[24] Almut Hupbach, Rebecca Gomez, Oliver Hardt, and Lynn Nadel. Reconsolidation of episodic memories: A subtle reminder triggers integration of new information. Learning & memory, 14(1-2)447-3,, 2007.
[25] Sang-Wor Woo Lee, Jin-Hwa Kim, Jachyun Jun, Jung-Wer Woo Ha, and Byoung- Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. Advances in neural information processing systems, 30, 2017.
[26] Hongjoon Ahn, Sungmin Cha, Donggyu Lee, and Taesup Moon. Uncertainty-baser continual learning with adaptive regularization. In Advances in Neural Information Processing Systems, volume 32, pages 4392 4402, 2019.
[27] Jie Zhang, Junting Zhang, Shalini Ghosh, Dawei Li, Jingwen Zhu, Heming Zhang, and Yalin Wang. Regularize, expand and compress: Nonexpansive continual learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 854 862, 2020.
[28] Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pages 3987-3995, PMLR, 2017.
[29] David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy P Lillierap, and Greg Wayne. Expe- rience replay for continual learning. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pages 350-360, 2019.
[30] Liyuan Wang, Xingxing Zhang, Kuo Yang, Longhui Yu, Chongxuan Li, Lanqing HONG, Shifeng Zhang, Zhenguo Li, Yi Zhong, and Jun Zhu. Memory replay with data compression for continual learning. In International Conference on Learning Representations, pages 1-25, 2022.
[31] Huiping Zhuang, Zhenyu Weng, Hongxin Wei, Renehunzi Xie, Kar-Ann Toh, and Zhiping Lin. Acil: Analytic slass-incremental learning with absolute memorization and privacy protection. In Advances in Neural Information Processing Systems, volume 35, pages 11602-11614, 2022.
[32] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Proceedings of the 31 International Conference on Neural Information Processing Systems, pages 1470-6479, 2017.
[33] Huan Liu, Li Gu, Zhixiang Chi, Yang Wang, Yuanhao Yu, Jun Chen, and Jin Tang. Few-shot class-incremental learning via entropy-regularieg data-free replay. In European Conference on Computer Vision, pages 146-162. Springer, 2022.
[34] Yoojin Choi, Mostafa El-Khamy, and Jungwon Lee. Dual-teacher class-incremental learning with fata-free generative replay. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 3543-3551 2021.
[35] Mozhgan PourKeshavarzi, Guoying Zhao, and Mohammad Sabokrou. Looking back on learned experiences for class/task incremental learning. In International Conference on Learning Representations, 2022.
[36] Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. In International Conference on Learning Representations, pages 1-11, 2018.
12

---

[37] Zixuan Ke, Bing Liu, Hao Wang, and Lei Shu. Continual learning with knowledge transfer for sentiment classification. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 683-698. Springer, 2020.
[38] Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, and Lei Shu. Achieving forgetting prevention and knowledge transfer in continual learning. In Advances in Neural Information Processing Systems, volume 34, pages 22443-22456, 2021.
[39] Hao Wang, Bing Liu, Shuai Wang, Nianzu Ma, and Yan Yang. Forward and backward knowledge transfer for sentiment classification. In Asian Conference on Machine Learning, pages 457-472.
PMLR, 2019.
[40] Ghada Sokar, Decebal Constantin Mocanu, and Mykola Pechenizkiy. Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 85-101. Springer, 2022.
[41] Zhiyuan Chen, Nianzu Ma, and Bing Liu. Lifelong learning for sentiment classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 750-756, 2015.
[42] Hao Wang, Shuai Wang, Sahisnu Mazumder, Bing Liu, Yan Yang, and Tianrui Li. Bayes- enhanced lifelong attention networks for sentiment classification. In Proceedings of the 28th International Conference on Computational Linguistics, pages 580-591, 2020.
[43] Paul Ruvolo and Eric Eaton. Ella: An efficient lifelong learning algorithm. In International conference on machine learning, pages 507-515. PMLR, 2013.
[44] Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. International Conference on Machine Learning, pages 4548-4457 PMLR, 2018
[45] Haeyong Kang, Jaehong Yoon, Sultan Rizky Hikmawan Madjid, Sung Hwang, and Chang D. Yoo. On the soft-subnetwork for few-shot class incremental learning. In The Eleventh Interna- tional Conference on Learning Representations, pages 1-23, 2023.
[46] Arjun Ashok, KJ Joseph, and Vineeth N Balasubramanian. Class-incrementee learning with cross-space clustering and controlled transfer. In European Conference on Computer Vision, pages 105-122, 2022.
[47] Zixuan Ke, Bing Liu, and Xingchang Huang. Continual learning of a mixed sequence of similar and dissimilar tasks. In Advances in Neural Information Processing Systems, volume 33, pages 18493-18504, 2020.
[48] Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang. Beyond not-forgetting: Continual learning with backward knowledge transfer. Advances Neural Information Processing Systems, volume 35, pages 16165-16177, 2022.
[49] Warren S McCulloch and Walter Pitts. A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5:115-133, 1943.
[50] Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 3014-3023, 2021.
[51] Gaurav Kumar Nayak, Konda Reddy Mopuri, Vaisakh Shaj, Venkatesh Babu Radhakrishnan, and Anirban Chakraborty. Zero-shot knowledge distillation deep networks. International Conference Machine Learning, pages 4743-4751. PMLR, 2019.
[52] Jiayu Lin. On the dirichlet distribution. Department of Mathematics and Statistics, Queens University, pages 10-11, 2016.
13

---

[53] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint 150.002333.
[54] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pages 2001-2010, 2017.
[55] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers features from tiny images. Master's thesis, University of Tront, 2009.
[56] Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/nnist/ 1998.
[57] A Kirzhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convo- lutional neural networks. In Advances neural information processing systems, volume 25, pages 1097-1105. 2012.
[58] Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient- bassen neural networks. arXiv preprint arXiv:1312.6211, 2013.
[59] Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, pages 1-42, 2019.
14


Reviews:
[{"review_id": "OB7FCPAVvl", "reviewer": "Reviewer_iEXK", "summary": "Building upon the principles of the Lottery Ticket Hypothesis, this paper introduces a hyper network model embedded with a series of competitive \"ticket\" sub-networks. Each of these sub-networks is designed to excel at their corresponding tasks, with a particular emphasis on knowledge transfer. Furthermore, this model promotes not just forward knowledge transfer, but also supports backward transfer of knowledge.", "questions": "N/A", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "Pros:\n1. The DSN method facilitates not only forward knowledge transfer but also backward transfer, offering a more dynamic and comprehensive learning paradigm that closely mimics human cognitive processes.\n\n2. The DSN method uses a neuron-wise mask and data-free memory replay, which can significantly save computational resources compared to maintaining a binary mask equal to the number of parameters.\n\n3. By using the mask to select neurons that have been used in earlier tasks and keeping their corresponding weights unchanged, the DSN method effectively addresses the issue of catastrophic forgetting that is common in continual learning scenarios.\n\n4. The data-free replay mechanism treats the trained subnetwork as a past experience and uses it to craft impressions of past samples, which bypasses the need to retain actual past samples, thus addressing privacy concerns and computational overhead.\n\n", "weaknesses": "Cons:\n\n1. The DSN method may involve complex processes and computations, such as measuring mask similarity scores and fine-tuning the most similar tasks, which might make the method computationally intensive and challenging to implement.\n\n2. While task-specific masks help in achieving better model performance, they add another layer of complexity to the training process and may lead to overfitting if not handled correctly.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## 1. Response to \"*complex processes*\"\n\n*(1) Similarity measurement*: Thanks for this concern. First, the mask similarity measurement is a simple vector-based computation that uses the cosine distance to obtain the similarity scores between different tasks. Second, our masks are neuron-level, which will further decrease the computation cost. For instance, given a task network containing a two-layer MLP and a classifier, each layer consists of $n$ neurons, yielding the neuron mask length for each layer is $n$. Thus, the measuring complexity between two tasks is $\\mathcal{O}(n)$ due to the element-wise computation. Instead, the complexity of the existing weight-wise mask mechanism will result in $\\mathcal{O}(n^2)$.\n\n*(2) Fine-tune the most similar task*: Thanks for this concern. First, the most important contribution of DSN is to enable knowledge transfer from the current task to the previous task (i.e., backward knowledge transfer). The main additional computation costs happen in this procedure due to the impression crafting and the most similar task fine-tuning. In practice, the complexity of the backward knowledge transfer has been considered during the model design investigation. Specifically, transferring newly learned knowledge to multiple similar old tasks can result in significant time costs (as well as memory costs) that outweigh the benefits. Thus, as we claimed in the paper, we only make backward knowledge transfer to the most similar task. According to Table 2, we can find that our proposed DSN does not cause a huge time consumption. Correspondingly, how to further compress the training time is the focus of our future research. In addition, compared to more recent state-of-the-art methods such as WSN, our number of parameters is significantly less than theirs, making the network more lightweight.\n\n*(3) Implementations*: In our Supplementary Material, our source codes were available and the implementation details were provided in the Appendix.pdf file.\n\n## 2. Response to \"*masks*\"\n\nThanks for this important concern. Several previous studies such as HAT and WSN demonstrate that the mask mechanism can be used to determine a subnetwork for each task from a large hypernetwork. Motivated by achievements in network pruning and Lottery Ticket Hypothesis, a hypernetwork is usually over-parameterized and usually even brings negative impacts on task performance than a smaller network. That is to say, many neurons or weights do not make any contribution to the task performance. In a continual learning context, ours and previous studies such as WSN consider that using the mask can discover a more compact subnetwork for each task. To bypass occasionality and randomness such as overfitting, our main experiments were compared by averaging the results of multiple training rounds (cf. Table 1)."}, {"review_id": "iKXcLKZGPV", "reviewer": "Reviewer_ai52", "summary": "This paper focuses on task continual learning and attempts to enhance the elastic knowledge transfer across the tasks that sequentially arrive. With the help of masks, achieve forward and backward knowledge transfer.", "questions": "Please refer to the Weakness part.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper is well written and easy to follow. The proposed method can achieve backword knowledge transfer with the help of mask similarity. ", "weaknesses": "1. Why can masks identify the similarity of tasks? The mask of the new task before learning should not be determined, so how to complete knowledge transfer at beginning.\n2. During the forward transfer, the parameters of the old task were fixed to prevent catastrophic forgetting, while during the reverse transfer, the parameters of the old task were optimized. The two seem somewhat contradictory. Can you unify the two to improve learning efficiency?\n3. The EWC and IMM algorithms used in the experiment are both class incremental learning. How can they be compared to Task incremental learning?\n4. The operation of convolutional layers is somewhat different from fully connected, and the paper lacks effective descriptions of other structures. Does the convolutional part regards each convolutional kernel as fully connected int the experiments?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## 1. Response to \"*mask & knowledge transfer*\"\n\n*(1) Forward Knowledge transfer*: Thanks for this question. Our solution is to use the mask mechanism to determine the subnetwork architecture of each arrived task from the hypernetwork $\\mathcal{H}$. Thus, any subnetwork is a subset of $\\mathcal{H}$. For a newly coming task $t$, the neuron-wise masks regarding $t$ are dynamically changed during the training process, as they are conditioned on the layer embeddings (cf. Eq.(3)). In other words, the embeddings corresponding to the masks are also trained during the task training process, and the final masks for the task $t$ are determined only when the training performance is optimal. As defined in Eq.(5), DSN using mask operation will only update the parameters that are not used in the previous tasks. And we regard the used neurons in the previous tasks as the synapses that only take the role of sending messages between different layers. Hence, the reused neurons from the past tasks allow the forward knowledge transfer to the new task.\n\n*(2) similarity tasks*: Due to the unavailability of old samples, we cannot measure task similarity using the approach of [1], which relies on the distribution of data from previous tasks. In our solution, we treat the subnetwork determined by mask mechanism as knowledge rather than data. Since the subnetworks all originate from the same hypernetwork, with the optimization of Eq. (8), we enforce the new task to reuse more old neurons. Therefore, we measure whether two tasks are similar or not by mask similarity, which is actually subnetwork structure similarity.\n\n[1] Ke Z, Liu B, Huang X. Continual learning of a mixed sequence of similar and dissimilar tasks. NIPS, 2020.\n## 2. Response to \"*backward knowledge transfer*\"\n\nSorry for this confusion. Above we have explained the procedure for forward knowledge transfer. Please allow me to explain the backward knowledge transfer here. For a newly coming task $t$, we will obtain its optimal masks after training convergence. As we claimed that the newly learned knowledge could be beneficial for the past tasks. Thus, in backward knowledge transfer, we first make the task similarity measurement by mask measurement (cf. Eq.(10)) to find the most similar old task (i.e., task argmax($S_t$)). Then, we employ our data-free replay to generate impression crafts. As claimed in the paper, these impressions are to determine whether we need to adjust the subnetwork architecture of the most similar task. During backward knowledge transfer, we will not change any parameters in the hypernetwork as it could cause interference with other tasks. But, we can update the masks of old task argmax($S_t$) as well as the task-specific classifier. That is to say, the subnetwork architecture regarding task argmax($S_t$) can be adjusted while keeping the parameters fixed. In this way, this operation will not cause interference with other old tasks as well as the newly coming task. Based on this interference concern, we cannot unify the two knowledge transfer procedures in our solution due to the sequential nature of the learning process.\n\n## 3. Response to \"*EWC & IMM*\"\n\nThanks for this concern. EWC[1] and IMM[2] have been widely adopted as the baselines for Task incremental learning such as [3][4][5].\n\n[1] Kirkpatrick, James, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan et al. \"Overcoming catastrophic forgetting in neural networks.\" *Proceedings of the national academy of sciences* 114, no. 13 (2017): 3521-3526.\n\n[2] Lee, Sang-Woo, Jin-Hwa Kim, Jaehyun Jun, Jung-Woo Ha, and Byoung-Tak Zhang. \"Overcoming catastrophic forgetting by incremental moment matching.\" *Advances in neural information processing systems* 30 (2017).\n\n[3] Serra, Joan, Didac Suris, Marius Miron, and Alexandros Karatzoglou. \"Overcoming catastrophic forgetting with hard attention to the task.\" In *International conference on machine learning*, pp. 4548-4557. PMLR, 2018.\n\n[4] Qin, Qi, Wenpeng Hu, Han Peng, Dongyan Zhao, and Bing Liu. \"Bns: Building network structures dynamically for continual learning.\" *Advances in Neural Information Processing Systems* 34 (2021): 20608-20620.\n\n[5] Masana, Marc, Tinne Tuytelaars, and Joost Van de Weijer. \"Ternary feature masks: zero-forgetting for task-incremental learning.\" In *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, pp. 3570-3579. 2021.\n\n## 4. Response to \"*convolutional layers*\"\n\nThanks for this concern. We depicted the complete architectures of CNN in Appendix C.3. Following HAT and WSN, our CNN is a modified version of AlexNet containing three convolution layers and two fully-connected layers. Indeed, the convolution layers are different from MLPs. In our experiments, we treat each filter in convolution layers like the 'neuron' in MLPs. And using the mask operation to determine which filter will be activated for the newly coming task training."}, {"review_id": "6UXAVp02rK", "reviewer": "Reviewer_bsXa", "summary": "The paper proposes a novel method for task incremental learning that uses a subnetwork for each task. The method is based on neuron masking obtained by learnable embedding. The method also finds the most similar task from the past tasks and allows for backward transfer. The method is tested on four benchmarks: PMNIST, RMNIST, CIFAR-100, and TinyImageNet. The results show an improvement in performance over other methods. ", "questions": "-   What is the contribution of each proposed components (neuron wise mask and data free replay) to the performance gain in DSN? Do you have an ablation for that?\n-   L 215 “Instead, we only allow the parameter update in the task-specific classifier (head) while freezing any parameters in the subnetwork.” This sentence is not clear. Does it mean that in backward transfer, you only update the head?\n-   Looking at Table 1 and Table 2 together and comparing WSN to DSN, do you think that the improvement gain in performance worths the additional costs (i.e., double the training cost in some cases). In other words, Are there specific cases in that one can definitely choose DSN over WSN?\n-   Is the proposed method applicable for more realistic scenarios i.e. class-incremental learning?\n-   Do you have some thoughts on how this approach can be further extended/improved to allow for backward transfer to multiple tasks?", "limitations": "-\tLimitations and social impact are not discussed", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-   The proposed method achieves a better performance than other methods on the three studied benchmarks.\n-   Up to my knowledge, few papers addressed positive backward transfer in CL. It is nice to see more work addressing this aspect.\n-   Analysis of the reusability of the neurons over tasks is provided.\n-   Efficiency analysis is considered.\n-   Different metrics are evaluated Accuracy, BWT, and Trans, which assess different requirements in CL.\n", "weaknesses": "-  Backward transfer to one only past task is not that convincing. Multiple previous tasks could benefit from the new knowledge if they are similar enough. This is my main concern. Since backward transfer is one of the main contributions, I expect more investigation into that point.\n-  Despite the fact that most of the paper is easy to follow, Section 3.3 lacks an overview at the beginning that can link the following information to how it contributes to the method. It becomes more clear after reading Section 3.4.\n\n [suggestion] Maybe the authors can consider reordering these two sections or clarify a bit more in Sec 3.3\n-  The paragraph explaining the backward transfer is very brief (Line 125-221), and some sentences are not clear. More elaboration could be useful. Algorithm 1 is helpful, though.\n-  An analysis that assesses the validity of the proposed similarity measure (even on a toy experiment) could be helpful.\n-  Minor: text in figures 3 and 4 is not readable.\n-  Minor: typo in the last line in Algo.1 “hyep”\n-  Some closely related works are missing. Not necessarily to empirically compare with them as you provide a comparison with one recent related method (WSN), but at least to have a discussion in the related work section. Examples of the missing works are below.\n\no\tGurbuz, Mustafa B., and Constantine Dovrolis. \"NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks.\" International Conference on Machine Learning. PMLR, 2022.\n\no\tSokar, Ghada, Decebal Constantin Mocanu, and Mykola Pechenizkiy. \"Spacenet: Make free space for continual learning.\" Neurocomputing 439 (2021): 1-11.\n\no\tSokar, Ghada, Decebal Constantin Mocanu, and Mykola Pechenizkiy. \"Avoiding Forgetting and Allowing Forward Transfer in Continual Learning via Sparse Networks.\" Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Cham: Springer Nature Switzerland, 2022.\n\no\tWang, Zifeng, et al. \"SparCL: Sparse continual learning on the edge.\" Advances in Neural Information Processing Systems 35 (2022): 20366-20380.\n\no\tYin, Haiyan, and Ping Li. \"Mitigating forgetting in online continual learning with neuron calibration.\" Advances in Neural Information Processing Systems 34 (2021): 10260-10272.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## 1. Response to \"*multiple previous tasks*\"\nThanks for this concern. We consider that transferring new knowledge to multiple old tasks can result in significant time costs (as well as memory costs) that outweigh the benefits. Thus, we only make backward knowledge transfer to the most similar task. Herein, we also provided experiments that DSN can transfer to more old tasks as below. We find that this version can further promote model performance. However, the time cost is significantly increased.\n| Dataset | Multiple old tasks | Accuracy(%) | BWT(%) | Elapsed Time |\n| --- | --- | --- | --- | --- |\n| PMNIST | No  | 98.24 | 0.01 | 2.43h |\n| PMNIST | Yes | 98.28 | 0.03 | 4.75h |\n| RMNIST | No  | 97.73 | 0.02 | 2.18h |\n| RMNIST | Yes | 97.88 | 0.06 | 3.95h |\n| CIFAR100 | No  | 75.17 | 0.02 | 1.21h |\n| CIFAR100 | Yes | 75.34 | 0.07 | 10.92h |\n| TinyImageNet | No  | 46.56 | 0.04 | 1.54h |\n| TinyImageNet | Yes | 46.61 | 0.06 | 8.78h |\n\n## 2. Response to \"*Sec. 3.3*\"\nThanks for your insightful suggestions. In the upcoming release, we will present an overview of this section. In addition, we will thoroughly revise all the typos and also engage in a comprehensive discussion of the related works.\n\n## 3. Response to \"*Ablation study*\"\nThanks for this question. In our forward knowledge transfer, the role of the neuron-wise mask is to select an optimal subnetwork architecture for the newly coming task. That is to say, it will affect the new task learning. As for data-free replay, it aims to produce impression crafts for backward knowledge transfer. Hence, this component will significantly affect the accuracy improvements of old tasks. In other words, it is directly related to backward knowledge transfer.\nWe provided the ablation study as follows.\n\n| Dataset | Neuron-wise mask | Data-free replay | Accuracy(%) |\n| --- | --- | --- | --- |\n| PMNIST | No  | Yes | 97.99 |\n|     | Yes | No  | 98.13 |\n|     | Yes | Yes | 98.24 |\n| RMNIST | No  | Yes | 97.42 |\n|     | Yes | No  | 97.65 |\n|     | Yes | Yes | 97.73 |\n| CIFAR100 | No  | Yes | 74.28 |\n|     | Yes | No  | 74.81 |\n|     | Yes | Yes | 75.17 |\n| TinyImageNet | No  | Yes | 46.02 |\n|     | Yes | No  | 46.41 |\n|     | Yes | Yes | 46.56 |\n\nFor the neuron-wise mask in the above table, if the choice is \"NO\", we use the weight-wise mask instead. For Data-free replay, if the choice is \"No\", we block this module. The above results demonstrate that removing the neuron-wise mask is more sensitive to the model performance, which also suggests that our neuron-wise mask is better than the weigh-wise mask. Moreover, we find that removing the data-free replay component also degrades the model performance, which demonstrates that DSN enables the knowledge transfer to the old tasks.\n\n## 4. Response to \"*L 215*\"\n\nSorry for the confusion. During backward knowledge transfer, we will not change any parameters in the hypernetwork as it could cause interference with other tasks. But, we update the masks of the most similar old task as well as the task-specific head. That is to say, the subnetwork architecture regarding the most similar old task can be adjusted while keeping the parameters fixed. In this way, this operation will not cause interference with other old tasks as well as the newly coming task.\n\n## 5. Response to \"WSN and DSN\"\n\nThank you for this great question. We have the following explanation.\nWSN mainly relies on the weight-wise mask to select the optimal subnetwork from a hypernetwork. In practice, the selected subnetwork is not a continuous structure of a layer-dependent neural network. This is because the WSN selects a portion of the weights that are relevant to the neuron. This results in the inability to separate the subnetwork selected from the hypernetwork into a single but complete network architecture. That is to say, every task in the continual learning system would need to download this huge hypernetwork in order to complete the model inference. In the future scenarios such as pervasive computing may be unrealistic. In addition, WSN needs to maintain a binary mask whose size equals the number of parameters also results in significant resource consumption.\n\nWSN and many other prior works have well addressed the CF problem, i.e., zero forgetting or forgetting-free, but they do not consider backward knowledge transfer well. That is, newly acquired knowledge cannot help facilitate any old task. In contrast, our DSN is capable of achieving positive gains on old tasks. Meanwhile, in the process of backward knowledge transfer, we also consider the data inaccessibility of the old tasks. And we use the data-free replay mechanism to effectively avoid this limitation.\n\nTherefore, we believe that our DSN offers a new paradigm that focuses more on positive backward knowledge transfer and takes into account the concern of data unavailability for old tasks.\n\n## 6. Response to \"*class-incremental learning*\"\n\nThanks for this question. In practice, our proposed DSN can apply to more contexts including class-incremental learning. [1] also gives a detailed theoretical study that task-incremental learning can be naturally transformed into class-incremental learning with the help of out-of-distribution detection.\n[1]Kim et al. A theoretical study on solving continual learning. NeurIPS 2022.\n\n## 7. Response to \"*future plan*\"\nThanks again, we plan to devise an efficient machinsm that aims to craft the representative impressions. Besides, we conduct backward transfer on past tasks in a sequential order, sorting them based on their similarity to the current task. We monitor the performance of past tasks after backward transfer. If their performance improves, we proceed to the next task in the sequence. We continue this process until the performance of the old tasks no longer shows enhancement from knowledge transfer. This iterative approach ensures that we prioritize and focus on transferring knowledge to the most relevant tasks first."}, {"review_id": "g4htwslYLb", "reviewer": "Reviewer_cKSd", "summary": "In this work, the authors explore task-incremental learning through the lens of the Lottery Ticket Hypothesis (LTH). They contend, primarily from an LTH perspective, that a distinct task necessitates merely a sparse collection of neurons, hence using only a compact sub-network for its operation. Subsequently, every new task can integrate itself within these compact sub-networks, thereby enabling the overall network to learn new tasks sequentially.\n\nThe methodology employed by the authors is intriguing. Initially, a hypernetwork is set up at random, followed by the gradual learning of the model's parameters in tandem with masks that are specific to each task. These masks play a crucial role, determining which neurons and corresponding weights should be utilized or rendered static for the impending task.\n\nNotably, the authors strive to enhance previously acquired knowledge via a process they term 'backward knowledge transfer'. This involves determining the most similar task previously encountered, creating a data impression of that task, and subsequently refining the mask/weights of the preceding tasks. This concept stands as a key contribution of this study.", "questions": "1. In connection with Objective 8, the authors introduce a capacity constraint to the current task, which may impact the task's performance due to the limitation it imposes on the neural network's capacity. I would be keen to hear an explanation from the authors on the extent to which this constraint influences the efficacy of the current task.  \n\n2. Considering the methodology used to measure the accuracy of task $argmax(S_t)$, I notice that the authors operate under the assumption of not having access to the previous task's data. Given this constraint, how do the authors determine the accuracy of the task post fine-tuning $argmax(S_t)$ and subsequently compare it with its preceding accuracy? Could they elaborate on their approach to evaluating accuracy under these specific conditions and how they ensure the validity of their comparisons?\n\n", "limitations": "In relation to the proposed method, I understand that tasks are stored as parts of a subnetwork. This is an intriguing concept, yet I find myself questioning its scalability when considering a large number of diverse tasks. Given that the total upper capacity of the neural network is fixed, I am uncertain if the proposed method would accommodate scaling up effectively. It seems to me that a broad and varied set of tasks may exceed the inherent capacity limits of the neural network, potentially leading to capacity exhaustion or compromise in task performance.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "\n1. The organization of the paper is commendable, effectively guiding readers through the progression of ideas, hypotheses, methodology, results, and conclusions. This well-structured approach allows for a clear understanding of the study and its outcomes.\n2. The concept of data-free backward transfer presented in this work is notably intriguing. This aspect, overlooked by existing methods, has been adeptly integrated into the authors' framework, contributing to its uniqueness and potential impact in the field.\n\n\n3. The authors have executed a compelling array of experiments in their work. Their holistic analysis of their method in relation to Forward Knowledge Transfer, Backward Knowledge Transfer, capacity issues, efficiency, and sensitivity analysis, is thoroughly detailed and insightful. This comprehensive evaluation underscores the robustness of their methodology and enhances the overall credibility of their findings.\n", "weaknesses": "1. The present work bears a significant similarity to the research conducted by Kang et al. [1], which also advocates for a similar framework leveraging the Lottery Ticket Hypothesis (LTH) and employs binary masks to learn task-oriented sub-networks. It would be advantageous for the authors to highlight their unique additions, modifications, or contributions vis-à-vis this recent analogous work, as this would underscore the true value and novelty of the current work.\n2. For each new subtask, the presented method learns a mask embedding corresponding to the task. The current manuscript would have been more complete if the authors had discussed the space complexity of the masked embeddings learned with respect to the tasks, comparing it to [1].\n\nMinor:\n1. A more concrete review/literature survey of data-free replay should have been performed, For example, somewhat related works like [2] and [3] have not been discussed.  Nonetheless, I encourage authors to explore more related works have utilized data-free replay in various other contexts.\n\n[1] Haeyong Kang, Rusty John Lloyd Mina, Sultan Rizky Hikmawan Madjid, Jaehong Yoon, Mark Hasegawa-Johnson, Sung Ju Hwang, Chang D. Yoo. Forget-free Continual Learning with Winning Subnetworks, In ICML, 2022.\n\n[2] Liu, Huan, Li Gu, Zhixiang Chi, Yang Wang, Yuanhao Yu, Jun Chen, and Jin Tang. \"Few-shot class-incremental learning via entropy-regularized data-free replay.\" In ECCV, 2022.\n\n[3] Choi, Yoojin, Mostafa El-Khamy, and Jungwon Lee. \"Dual-teacher class-incremental learning with data-free generative replay.\" In CVPR Workshops, 2021.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## 1. Response to \"*WSN*\"\n\nThanks for this suggestion.\n\n(1) DSN devises a neuron-wise mask mechanism to select neuron-affiliated weights for new task learning.\n\n(2) DSN enables positive knowledge transfer in both forward and backward directions. In particular, the data-free replay mechanism in DSN regards the trained subnetwork as a past experience and uses it to craft impressions regarding the past samples, which does not require holding any actual samples related to past tasks.\n\nThe comparison details:\n(1) WSN mainly relies on the weight-wise mask to select the optimal subnetwork from a hypernetwork. But the selected subnetwork is not a continuous structure of a layer-dependent neural network. Because WSN selects a portion of the weights that are relevant to the neuron. WSN also maintains a binary mask whose size equals the number of parameters, resulting in significant resource consumption. In contrast, DSN devises a neuron-wise mask to choose the optimal subnetwork, which can ensure the completeness of the subnetwork. Meanwhile, since our masks are neuron-level, they are lightweight. According to Table 3, we can find DSN requires less number of masks.\n\n(2) WSN has well addressed the CF problem, but WSN does not incorporate backward knowledge transfer. That is, newly acquired knowledge cannot help facilitate any old task. In contrast, DSN is capable of achieving positive gains on old tasks. Meanwhile, in backward knowledge transfer, we consider the data inaccessibility of the old tasks. And we use the data-free replay to effectively avoid this limitation.\n\nThus, DSN is significantly different from WSN. We believe that DSN offers a new paradigm that focuses more on positive backward knowledge transfer and consider the concern of data unavailability.\n\n## 2.Response to \"*space complexity*\"\n\nFor instance, given a task network containing a two-layer MLP and a classifier, each layer consists of n neurons, yielding the length of neuron mask embedding for each layer is $n$. Thus, the space complexity of each is $\\mathcal{O}(n)$ due to its size equal to the neuron numbers. Instead, WSN will result in $\\mathcal{O}(n^2)$. According to Table 3, we can find that DSN requires less number of masks.\n\n## 3. Response to \"*minor*\"\n\nThanks for your significant suggestions. In the next release, we will thoroughly revise all the typos and also engage in a comprehensive discussion of the related works.\n\n## 4. Response to \"*Objective 8*\"\n\nThanks for this question. In Eq.(8), we set a hyperparameter $\\eta$ to control the capacity preference for each new task. The higher the value of $\\eta$, the lower the number of neurons activated for the new task. In particular, there is no capacity limit when $\\eta =0$.\n\nFirst, in our main experiment, as shown in Fig. 6, when $\\eta =0$, it is evident that all neurons have been utilized. But, the accuracy performance is the worst. Obviously, it is because the model cannot utilize new neurons to learn new knowledge as more new tasks arrive. In addition, a larger $\\eta$ indicates that the model will hold more room for new task learning. However, we have found that having an excessively large capacity constraint does not lead to a significant improvement in accuracy performance. In other words, excessive reuse of old neurons can also result in underfitting issues during training for new tasks.\n\nSecond, in Appendix D.2, we conducted another experiment where we set up a different number of tasks (up to 100 tasks) to validate the performance, where the hypernetwork settings were the same as in the main experiment. As Fig. 10 shows, compared to HAT and WSN, we first observe that the accuracy of all three methods decreases as the number of tasks increases, mainly due to the fact that the hypernetwork is too small. However, we observe that the DSN always has the best performance as the number of tasks increases.\n\n## 5. Response to \"*old task accuracy*\"\n\nThanks for this question. As we consider the unavailability of old samples regarding task argmax($S_t$), we are motivated by zero-shot learning and treat the trained subnetwork as the past knowledge of task argmax($S_t$). We also have it determined output space. Thus, we can craft the input space of task argmax($S_t$) using data-free replay. As these impression crafts are determined by the network of the old task, we use them to obtain ground-truth accuracy. Then, we merge the optimal mask of the current task to the most similar task. This suggests that the old task is able to share neurons used in new tasks, enabling the transfer of new knowledge. Next, we use impression crafts to update the mask of the old task. DSN decides to maintain the updated mask for the old task only if the updated subnetwork shows a significant improvement in accuracy over the ground-truth.\n\n## 6. Response to \"*capacity limitation*\"\n\nThanks again. As Fig. 4 and Fig. 11 show, with the number of tasks increases, the network does not reach its limit, indicating that DSN effectively utilizes acquired knowledge to handle new tasks. Additionally, we conduct additional experiments to evaluate whether a progressive network can improve accuracy. We enable a random expansion of neurons when a new task arrives. The results indicate that the model does not significantly benefit from expansion. Besides, the expansion will lead to a substantial increase in parameters, as demonstrated below. Your insightful suggestion shad light on a new perspective. We will explore how to effectively utilize the newly added neurons in future work.\n\n| Data | Fixed | Accuracy(%) | Layer1 | Layer2 | Layer3 |\n| --- | --- | --- | --- | --- | --- |\n| PMNIST | Yes | 98.24 | 2000 | 2000 | N/A |\n|     | No  | 98.29 | 2273 | 2331 | N/A |\n| RMNIST | Yes | 97.73 | 2000 | 2000 | N/A |\n|     | No  | 97.75 | 2256 | 2385 | N/A |\n| CIFAR100 | Yes | 75.17 | 64  | 128 | 126 |\n|     | No  | 75.21 | 354 | 440 | 509 |\n| TinyImageNet | Yes | 46.56 | 64  | 128 | 256 |\n|     | No  | 46.58 | 391 | 345 | 576 |\n"}, {"review_id": "zrMeZs6vVu", "reviewer": "Reviewer_Um41", "summary": "This paper presents a Data-free Subnetworks (DSN) approach for task incremental learning. With a random initialized neural network, DSN learns a task-specific neuron-wise mask to find an optimal subnetwork for a new arriving task, and performs data-free replay for transferring the knowledge to the past tasks. DSN achieves superior performance on four benchmarks.", "questions": "- Which direction(s) does the proposed method belong to, regularization-based, rehearsal-based, or architecture-based? \n- What does the f represent in Eq. (1) and (2)?\n- What is the difference between a neuron-wise mask and a weight-wise mask?\n- What does “bridge the gap between the layer embedding and layer mask” mean?\n- Could you provide more details about the structure of the over-parameterized deep neural network used in this method, such as the number of layers and neurons in each layer?\n- In Algorithm 1, what do B1, ..., B_{argmax(S_t)} represent?", "limitations": "Yes. The authors have discussed the limitations regarding the efficiency and capacity issue of the proposed method. ", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The paper's approach of designing a task-incremental learning method motivated by the Lottery Ticket Hypothesis is reasonable and novel.\n- The emphasis on backward knowledge transfer, often overlooked in previous incremental learning methods, is commendable.\n- The proposed method outperforms many State-Of-The-Art methods across four benchmarks under multiple evaluation metrics, demonstrating its effectiveness.", "weaknesses": "- It is confusing how backward knowledge transfer is achieved through data-free replay. By definition, backward knowledge transfer refers to the process in which new knowledge is transferred to old tasks to potentially improve their performance. However, it seems that the replayed samples (impression crafts) only represent knowledge of old tasks and may not incorporate new knowledge for improving performance in those tasks. \n\n&nbsp;\n- The related work section can be further improved by providing detailed comparisons with 1) architecture-based methods (such as [39][41]) that also learn masks to designate different subnetworks for different tasks; 2) incremental learning methods [40][1*] that also consider backward knowledge transfer; and 3) [2*] that is also motivated by Lottery Ticket Hypothesis.\nRef. \n[1*] Class-incremental learning with cross-space clustering and controlled transfer, ECCV 2022;\n[2*] On the Soft-subnetwork for few-shot class incremental learning, ICLR 2023\n\n&nbsp;\n- The method details and formulas need clarification. For example, Eq. (9) lacks an explanation of how to calculate L_{IC}, the optimization objective does not include the variable x, and the discussion on the validity of the Dirichlet distributions representing the task output is less convincing.\n\n&nbsp;\n- Knowledge transfer is only performed between current task and the most similar previous task. Considering more similar tasks and evaluating the associated cost could potentially lead to improvements. \n\n&nbsp;\n- Some minors: 1) Case errors, such as “Without loss of generality, A classical TIL scenario, ...”; 2) Brackets missing on line 185; 3) Fig. 6 is not referred to in the manuscript.\n\n&nbsp;", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## 1. Response to \"*backward knowledge transfer*\"\n\nThanks for this concern. After training a new task and task similarity measurement, our data-free replay is to produce impression crafts of the most similar task. For backward knowledge transfer, as we claimed that we treated the subnetwork as the knowledge instead of samples, we thus merge the optimal mask of the current task to the most similar task (cf. Knowledge Transfer in Sec. 3.4). This suggests that old task is able to share neurons used in new tasks, enabling the transfer of new knowledge. we then use the impression crafts to update the mask of the old task. This process ensures that the old task's mask is adapted to accommodate the new knowledge, effectively facilitating the transfer of valuable information from the new task to the past tasks. Thus, impression crafts serve as a crucial bridge for carrying over and integrating the latest knowledge into the existing model, leading to improved performance and informed adaptations across the entire task sequence.\n\n## 2. Response to \"*related work*\"\n\nThanks for this suggestion. [1*] proposes two distillation-based objectives for class incremental learning. This method can utilize the feature space structure of the previous model to preserve the representation of previous classes. Besides, a controlled transfer is introduced to approximate and condition the current model on the semantic similarities between incoming and prior classes, maximizing positive forward transfer and minimizing negative backward transfer. However, it can not achieve a positive backward transfer. [40] analyzes the conditions under which updating the learned model of old tasks can be beneficial and lead to backward knowledge transfer. It should be noted that this method is based on SVD, which leads to high computational costs for large dimensional data. [2*] is similar to WSN [3]. Nevertheless, it focuses on a few-shot scene, which is accomplished by jointly learning model weights and adaptive non-binary soft masks, with the major subnetwork minimizing catastrophic forgetting and the minor subnetwork preventing overfitting. Both [2*] and [39] fail to take backward knowledge transfer into consideration. The major contribution of [41] is the induction of the biological neural system into the \"M-P neuron model\", which sheds light on utilizing the neural network to handle complex tasks.\n\n## 3. Response to \"*Eq.(9)*\"\n\nThanks for this concern. $\\mathcal{L}_{IC}$ is the common cross-entropy loss.\n\n## 4. Response to \"knowledge transfer\"\n\nThanks for this concern. The most important contribution of DSN is to enable knowledge transfer from the current task to the previous task (i.e., backward knowledge transfer). The main additional computation costs happen in this procedure due to the impression crafting and the most similar task fine-tuning. In practice, the complexity of the backward knowledge transfer has been considered during the model design investigation. Specifically, transferring newly learned knowledge to multiple similar old tasks can result in significant time costs (as well as memory costs) that outweigh the benefits. Thus, as we claimed in the paper, we only make backward knowledge transfer to the most similar task. According to Table 2, we can find that our proposed DSN does not cause a huge time consumption. Also, we provided the additional experiments that the DSN can transfer to more old tasks **(cf. the Response 1 of #Reviewer bsXa**). We can find that this version can further promote the model's performance. However, the time cost is significantly increased.\n\n## 5. Response to \"*some minors*\"\n\nThanks for pointing out these minors, we will revise them in the next release.\n\n## 6. Response to \"*directions*\"\n\nStrictly speaking, our approach DSN falls under architecture-based. Unlike rehearsal-based solutions, we do not retain any samples from the old tasks.\n\n## 7. Response to \"*f*\"\n\nThanks for this concern. $f$ denotes a heypernetwork without any mask.\n\n## 8. Response to \"*weight-wise mask & neuron-wise mask*\"\n\nThanks for this question. The weight-wise mask is one in which each weight in the hypernetwork is associated with a mask value that determines whether or not that weight is used. Note that the size of weight-wise mask equals the number of parameters. In contrast, neuron-wise mask is one in which each neuron in the hypernetwork is associated with a mask value that determines whether or not that neuron is used.\n\n## 9. Response to \"*layer embedding & layer mask*\"\n\nThanks for this question. The layer embedding is the trainable vector. And each mask value is the score within the range of 0 to 1, which numerically determines whether the corresponding neuron should be activated.\n\n## 10. Response to \"*network details*\"\n\nThanks for this concern. We have presented the details of the network architecture in Appendix C.3, as shown below. \n\n- MLP for PMNIST and RMNIST: We follow [39] and start with 784-2000-2000-10 neurons with RELU activation.\n  \n- CNN for CIFAR-100 and TinyImageNet: We also follow [39] and extend a modified version of AlexNet for the first task. In more detail, it has three convolutional layers with 64, 128, and 256 filters, with 4×4, 3×3, and 2×2 kernel sizes, respectively, and plus two fully-connected layers of 2048 neurons each. Also, we use rectified linear units as activation and utilize a 2×2 max-pooling operation after three convolutional layers.\n\n## 11. Response to \"Algorithm 1\"\n\nSorry for this confusion, according to the class definition in **Output Space Modeling**, herein, each $B_i$ regarding $C_i$-th of task argmax$(S_t)$ refers to the number of impressions, where $C_{argmax (S_t)}$ indicates that the task argmax$(S_t)$ has $C_{argmax (S_t)}$ classes."}]

Author rebuttal:

Meta-review: This paper proposes a new approach to task-incremental learning called "Data-free Subnetwork" (DSN) which is a masking based method. The paper also proposes a data-free replay method to update the masks of previous tasks after learning new tasks. This enables backward transfer.

There were a lot of questions from the reviewers about the methodology. Authors clarified everything in the rebuttal. I do not think the authors need to apply their algorithm to class-incremental learning.

Given there is no major concern about the work, even from the low rating reviewers, I recommend an accept.

I recommend the authors to incorporate all their explanations and additional experiments into the final version of the paper.
Decision: Accept (poster)