Multiple Modes for Continual Learning
Authors: Siddhartha Datta
Nigel Shadbolt
Venue: Submitted to ICLR 2023 (2023)

Abstract: Adapting model parameters to incoming streams of data is a crucial factor to deep learning scalability. Interestingly, prior continual learning strategies in online settings inadvertently anchor their updated parameters to a local parameter subspace to remember old tasks, else drift away from the subspace and forget. From this observation, we formulate a trade-off between constructing multiple parameter modes and allocating tasks per mode. Mode-Optimized Task Allocation (MOTA), our contributed adaptation strategy, trains multiple modes in parallel, then optimizes task allocation per mode. We empirically demonstrate improvements over baseline continual learning strategies and across varying distribution shifts, namely sub-population, domain, and task shift.

Under review as a conference paper at ICLR 2023
MULTIPLE MODES FOR CONTINUAL LEARNING
Anonymous authors
Paper under double-blind review
ABSTRACT
Adapting model parameters to incoming streams of data is a crucial factor to deep learning scalability. Interestingly, prior continual learning strategies in online settings inadvertently anchor their updated parameters to a local parameter subspace remember old tasks, else drift away from the subspace and forget. From this observation, we formulate trade-off' between constructing multiple parameter modes and allocating tasks per mode. Mode-Optimized Task Allocation (MOTA), our contributed adaptation strategy, trains multiple modes in parallel, then optimizes task allocation per mode. We empirically demonstrate improvements over baseline continual learning strategies and across varying distribution shifts, namely sub- population, domain, and task shift.
INTRODUCTION
As the world changes, must our models of it. The premise of continual (or incremental lifelong) learning to build adaptive systems that enable a model to return accurate predictions as the test-time distribution changes, such change in domain or task. Training sequentially multiple different task distributions tends to result in catastrophic forgetting (McCloskey & Cohen 1989), where parameter updates benefiting the inference of the new task may worsen that of prior tasks. Alleviating this is the motivation for our work.
To enable flexibility in adoption, do not assume parameter adaptation with respect task-specific information, and we assume only access to model parameters alone (no conditioning inputs, query sets, rehearsal or replay buffers, -sshot metadata, or any historical data). This carries positive implications for adoption in online learning settings, and robustness towards different distribution shifts (e.g. sub-population, domain, task shifts). Interestingly, prior work in non-rehearsal methods (notably regularization and parameter isolation methods) tend to "anchor" the parameter updates with respect to a local parameter subspace. These methods begin with a model initialization, then update the model with respect the first task, and henceforth all future parameter updates new tasks computed with respect to this local subspace (usually minimizing the number of parameter value changes). The key question we ask here: what happens when consider the global geometry of the parameter space?
Our pursuit of adaptation method leveraging global geometry is supported by various initial observations. When learning tasks ..... T, a multi-task learner tends to drift large distance away from its previous parameters optimized for 1, ..., T indicating that when given information on all prior tasks, a multi-task learner would tend to move to a completely different subspace (Figure Mirzadeh al. (2020). Catastrophic forgetting intricately linked to parameter drift: unless subspace, if the new parameters drift further from the old parameter subspace, then accuracy is expected to drop for all prior tasks; not drifting sufficiently will retain performance on prior tasks, but fail on the new task. Coordinating parameter updates between multiple parameter modes tend keep the average parameter drift distance low (Figure 3
drifting towards multi-task-opile
Contributions. Grounded on these findings, we introduce new rehearsal-free continual learning algorithm (Algorithm []. We initialize pre-trained parameters, maximize the distance between the parameters the first task, then on subsequent tasks we optimize each parameter based on the loss with respect to their joint probability distribution as well as each parameter's 's drift from its prior position (and reinforce with backtracking). Evaluating forgetting per capacity, MOTA tends to outperform baseline algorithms (Table and adapts parameters to sub- populatiin,,, domain, and task shifts (Tables

---

Under review as a conference paper at ICLR 2023
taxonomized continual learning algorithms into replay, reg- ularization, and parameter isolation methods. Replay (or rehearsal) methods store previous task samples supplement retrainnn with the new task, such as iCaRL (Rebuffi et al. 2017). ER (Ratcliff) 1990: Robins 1995| Riemer et al. 2018 Chaudhry et al. 2019), and A-GEM (Chaudhry et al.
Related Work. Lange et al. (2019)
20T8b). Regularization methods add regularization terms the loss function to consolidate prior task
knowledge, such as EWC (Kirkpatrick et al., 201 7b). SI (Zenke et al. 2017), and LwF (Li & Hoiem 2016). These methods tend to rely on no other task-specific information or supporting data other than the model weights alone. Parameter isolation methods allocate different models or subnetworks
within model to different tasks, such as PackNet (Mallya & Lazebnik] 2017), HAT (Serrà et al.
2018 SupSup (Wortsman et al. 2020), Batchensemble (Wen et al. 2020), and WSN (Kang et al. 2022 Task oracles may be required to activate the task-specific parameters. Ensembling strategies
in this category may either require task indices to switch to specific task model (e.g. Wenn et al.
2020
) or update all ensemble models on all tasks but risk losing task-optimal properties of cach
parameter's subspace (e.g. Doan et al. (2022)
The loss landscape changes after each task (Figure Prior work either anchors to the local subspace of the first task, anchors each task to its specific local subspace, anchors the entire parameter space to the last seen task. We are the first to leverage the global geometry of a loss landscape changing with tasks without compromising the task-optimal properties of each subspace nor requiring any task-specific information.
2 TRADE-OFF BETWEEN MULTIPLE MODES AND TASK ALLOCATION
First we introduce the problem set-up of continual learning, with assumptions extendable to broader online learning settings. Then we share the observations that motivate our study into multiple modes. Finally we present a trade-off, which motivates our proposed learning algorithm.
Problem Setup. A base learner receives T tasks (or batches) sequentially. D. = (Ix,, yt) denotes the dataset of the t-th task. In the continual learning setting, given loss function L, neural network {0; z) optimizes its parameters 0 such that it can perform well on the t-th task while minimizing
performance drop on the previous (t 1) tasks: H* := arg ming Ei= C()(0;22)),0 yr). We assume the only information available at test-time is the model(s) parameters and the new task's data points. The learner cannot access any prior data points from previous tasks, and capacity is not permitted to increase after each task. Additionally, we do not assume parameter adaptation at test-time can be conditioned on task boundaries conditioning inputs (task index, replay, K-shot query data, etc).
methods are grounded on minimizing drift (change in parameters) to reduce forgetting on prior tasks. Yet in Table 3 a multi-task learner has a higher average drift consistently between tasks than EWC, even when both begin from a shared starting point (init - task 1). Given visibility to prior tasks, a multi-task learner will depart the sub- space of the previous parameter, and drift far. This contradicts with the notion of forcing parameters to reside in the subspace of the previous parameter. The ingularization-base, methods essentially anchor all future parameters to the first task observed.
Trade-off. Many regularization-
Results in mode connectivity (Garipov et al. 2018 Fort & Jas- trzebski 2019 Draxler et al., 2019 show that a single task can have multiple parameters ("modes") that manifest functional diversity. We explored computing multiple modes with respect to task 1 to incorporate the broader geometry of the parameter space beyond the subspace of one mode. To bring performance gains and capacity efficiency, we further obtained this trade-off between the number of modes, number of tasks allocated per mode, and capacity (Theorem 1).
Figure 1: A diagram of the differ- ent parameter trajectories demon- strating that, rather than anchoring all subsequent learning on mode O,, we can leverage the func- tional diversity of other modes for optimal task allocation.
We denote ginit as the initialization parameter, 0 (MT.().....) as the multi-task parameter trained on tasks 1 1.... T, and 0rr as the parameter of mode index i updated task t.
2

---

Under review as a conference paper at ICLR 2023
Theorem 1 If the number of modes N is optimized against capacity |0] and the set of tasks allocated per mode [T()) = {t}] for i E N, T, then the total task drift is lower in the multi-mode setting
than single-mode setting:
N 2 10
T()
101N/ (0.,,. m² -1=2 [0] dd1 (01,..
0 ya
drift (multi-moce
drift (single-mon mode)
Proof. See Appendix A.3
As result, our proposed algorithm is motivated to train multiple modes while optimizing for tasks learnt per mode. Our total capacity cannot exceed that of our baselines. Our improved performance in the continual learning setting also empirically validates Theorem 1.
3 MODE-OPTIMIZED TASK ALLOCATION
-rodi-Optimized Task Allocation (MOTA) To implement our method, there are two components: (i) initialize sizeeparameters initializing N modes/parameterss and (ii) update- update-parameters parameters updating a subset of modes/parameters. Once the first task is received, we train N models in parallel such that the distance between them maximized. Then for each subsequent task, we coordinate the parameter updates of the modes such that the drift per mode is minimized, and only the minimum number of modes needed to solve the task will updated.
3.1 MODE INITIALIZATION
We begin with pre-trained initialization (used for MOTA well as all baselines). We instantiate N models on this initialization of a fixed model architecture, and denote this set of parameters { for E N and & T. We train the parameters in parallel such that the distance between each other is maximized. For each batch per epoch, we randomly generate weights {a;} } that sum to I and compute an interpolated parameter 0 ΣN a, We compute the distance between the N modes E dist(f,,,. (),1) be maximized (adjusted with penalty coefficient 3max). We update each mode with respect to the input loss (evaluated with 0) and the distance maximization term. For this distance maximization procedure, we used the average cosine similarity between each layer E MM N -
05.0
- [Tşşll li5 out of and layers and models respectively. Our coordination of distance maximization between a set of parameters is Dntta
between every pair models dist (N)= >3
in-line with the methodology in WWotsman et al. (2021) and Datta & Shadbolt (2022). though their cases specify a unique random initialization per mode.
3.2 MODE ADAPTATION
The objective is to strategically update the parameters of a subset of modes required at the 1-th task such that we minimize the overall drift from each mode's prior state but infer accurately on the task t. For each mode per epoch, we compute the loss with respect to a joint probability distribution. We also compute distance term between each mode's 's parameters and its parameters at the (t - 1) task dist(O;,, 0,,-1) be minimized (adjusted with penalty coefficient 3 We use the EWC regularization term for distance minimization. We update each mode with respect to the joint loss and its respective parameter drift, and checkpoint each update. We iterate through the modes sequentially per epoch to minimize memory requirements for parallelized training.
We compute the gradient update for each mode with respect to the joint probability distribution between all the modes. Specifically, we compute the average probability distribution returned at the last (softmax) layer f = -1 of each model ) N EN ((0(=-); If task has a high level of certainty, then only a small subset of models would need to be updated and return a probability distribution skewed toward the target class while the other non-updated/ / minimally-updan would return a random distribution, and the resulting averaged distribution would still be slightly skewed towards the target class. For task of low certainty, then more models (of high functional diversity) would be updated to return a robust probability distribution. Furthermore, ensemble learning usually requires each ensemble model be trained independently (and usually with different
1
models
3

---

Under review as a conference paper at ICLR 2023
Algorithm I poodate-parameeess
procedure update_parrt parameerrprrm (0r,))) (
Pass a new task Dr to our cureem parameteers (0,,c) 3 Dr Check iniializin ng parsmettrs for first
2:
t=1 then
3:
(0c))i=1 +- initialize - (1)))==))
4: else
5:
{0,,-1]im1 J.v - (0,,,)=11 for in epochs do for 8r,, in (0,,) do
P Retain a of the last task's parameters
6:
7:
8:
for (1 Vt in D, do P10..) 1- = irant_inferencerry (0s,))i=)) )i=1)
9:
10:
L. = 1x, ) + Amin dist(fl.., 02,1-1)
Compute probability and
02.1 = 02, 86.
ahe
Update each parameter independently
12:
(t" = arg min L ",
Dr Backtmekkngg Emumerate through pammeter combinations
D dist(Oin, Oie-1) for (0i,,e -(0 Aimeefim
xepochs
13:
(0t))=11 t-
14:
return (0
Algorithm 2 initializeepprrmmterss
1: procedure initialize ID (mr,)
Di Initialize Dy empty parameters (f I
2: [0,,e] < {ginir] N
Ji=1
3: for in epochs do
4: for (æt, Vr in DD do
5:
a [0, E s.t. S.N 1==10, 022 1
6:
0 = I a:0i.
Sample interpolated paramete
7:
C. ((f(0); Tzg 31) + Bmax - dist(Os,e, 0g.z)
D Compute loss and distance term
0,e = 0, BC,
8:
20,
Update each independently
9:
return 1
Algorithm 3 joint_inferenc
1: procedure joint_ ientttiiffeeeneee..m 10
>> Inference the (0,,,)
2: PfO..... N 1f 10 ;x)
Taking average juint probobility distribution returned layer
return P(o....) 3:
initialization) and the final predictions are obtained as an average of the predictions of each model. Averaging procedures during inference (e.g. averaging activations, averaging softmax) are not used during training, as each ensemble model should not influence the gradient computation of another, and all ensemble models are expected to be trained on all tasks.
We checkpoint gradient updates per epoch, and can thus further optimize the sequence of mode updates by backtracking. Despite optimizing with respect to the joint probability distribution, one risk remains where minimum number of gradient updates across all modes need to take place before a stable joint probability distribution can be computed. In other words, by the time we have jointly-accurate modes, it is likely most of the modes have been over-optimized, and thus drifted more than needed. Thus, we need to backtrack and find the optimal checkpoints across modes that minimize loss with respect their checkpointed joint probability distribution and drift. We adopted the simplest backtracking algorithm: we enumerate through every combination of model checkpoint per epoch across the modes, and select the checkpoint combination that minimizes the loss with respect to the joint probability distribution for task t and minimizes total parameter drift. To reduce the propensity of selecting earlier-epoch checkpoints (e.g. non- non-updated models), we can add a penalty term to distance regularization to reduce its relative weighting input loss. This helps mitigate the loss imbalance between the input loss term and distance term for earlier checkpoints.
4

---

Under review as a conference paper at ICLR 2023
4 EVALUATION
We state our experimental setup below, with configuration details in Appendix A.1 our results on MOTA's 's improvement in task adaptation.
We then review
Incremental Learning (IL) Settings. task is a subsequent training phase with new batch of data, pertaining to new cobb-opulationddmmaiin new label set, or different output space. In aastance-LL, each new task bring new instances from known classes. In class/task-I1, each new task bring instances from new classes only. Class-IL performs inference w.r.t. all observed classes. Task-IL performs inference w.r.t. the label set of the task. We evaluate on task-IL unless otherwise specified.
Datasets. Task-IL Split-CIFAR100 (Krizhevsky) 2009) is con- structed by dividing 100 fine labels into 10 tasks (10 fine labels task). Instance- Split-CCAA 100 (Krizhevsky 2009) is constructed by dividing 100 fine labels (mapped to 5 coarse labels) into tasks (20 coarse labels per task). Instance-IL. DomainNet (Peng et al. 2019) composed of domains with 345 labels. Task-IL Tiny ImagNNet (Stanford) is constructed by dividing 200 labels into 10 tasks (20 labels per task).
0.74
Single-Task Learning
0.72
EWC
0.70
0.68
MOTA Multi-Task Learning
0.66
0.64
4 0.62
0.60
0.58
0.56
& 0.54
Architectures. We initialize ResNets (He al.) 2015). loading ImageNet-pretrained weights from Py Pyrorch. We train ResNet-18 (11, 181, 642 parameters). ResNet-50 (23, 528, 522 parameters), and ResNet-152 (58, 164, 298 parameters). To re- tain comparable capacity, ResNet-50 (-18) is the default model for baselines (MOTA). We do not assume task boundaries at test-time for parameter adaptation, and do not use task index to recompute task-specific parameters. We evaluate against reg- ularization methods (Elastic Weight Consolidation (Kirkpatrick) al.] 2017b), Synaptic Intelligence (Zenke et al. (017)). Learn- ing without Forgetting (Li & Hoiem (0166). We also compare against four replay baselines (Experience Replay (Rolnick et al.
0.52
0.50
0.46
2
3
4
Task
(a) Instance-IL.
0.80
0.78
0.76
0.74
0.72
0.70
0.68
2018). Averaged Gradient of Episodic Memory (Chaudhry et al.
Averag Accure
0.66
0.64
2018b), Gradient Projection Memory (Saha et 2021), and
0.62
La-MAML (Gupta et al. 2020 though they require task replay buffer for parameter adaptation. We further evalu- ate against two parameter isolation baselines (BatchEnsemble (Wen et al., 2020), and Winning SubNetworks (Kang al.. 2022)). We introduce two ablations: Ensemble (distance max.) which an ensemble of modes obtained using MOTA's distance maximization procedure and trained all tasks, and Ensemble (independent seeds) which is an ensemble of modes trained on all tasks but from independent random initializations.
0.60
0.58
0.56
0.54
0.52 Single-Task Learning
0,50 EWC
0.48 MOTA
0.46 Multi-Task Learning
2
4 5 7
9 10
Task
Metrics. Single-Task Learning trains on each task indepen- dently. Multi-Tash Learning trains on all seen tasks simulta- neously. Primarily basclined against regularization methods, capacity is the number of trainable model parameters. When considering replay methods as well, we distinguish capacity w.t.t model parameters from repay buffer, where the replay budget 100 per class. The average drift distance between tasks is the distance between the updated parameters and previous parameters, aver- aged for each task update instance. For multiple model parameters, we take the cumulative distance. --1 E IN bist(ff.,, 0.. Given Acc(O, 2x) as the validation accuracy on the /-th task, average accuracy is the average validation accuracy across all seen tasks w.r.t. the parameters updated at the 1-th task. We compute this from & Σ Acc(Oz, x.). Backward Transfer (Lopez-Paz Ranzato 2017) measures the influence that learning task has on the performance on previous tasks. We compute this from (Lopez-Paz Ranzato 2017) measures the influence that learning task has on the performance
(b) Task-IL
Figure 1: Per-task average accuracy
We compute this from 1
- Σ Acc(O,, (o(00,,,,) ...)) ) Acc(O, v.)). Forward Transfer
of future tasks. We compute this from ト ASc(2Acc((,__))) x..) Acc(ginit xx).. Remembering (Diaz-Rodriguez et al. 2018) computes the forgetting part of Backward Transfer. We compute this
5

---

Under review as a conference paper at ICLR 2023
from min(0, - Σ Acc((),, - (.)() Forgetting Chaudhry et al. 201 is
calculated by the difference of the peak accuracy and ending accuracy of each task. We compute this from
T-1 v=1 max ( (I, (Acc(fe, (o) acc(0r,2x.))
4.1 EVALUATING CATASTROPHIC FORGETTING
We evaluate MOTA against different types of distribution shifts (Table Figure 1 Table Evalu- ating on task shift in CIFARI00 and TinyImageNe we observe improved backward and forward transfer with MOTA, indicating lower forgetting well as improved feature transferability between tasks. As task-specific information is not necessary for parameter adaptation, we can evaluate our method on settings that do not require an assumption task boundaries or indexing, namely sub- population and domain shift. We find that our method also outperforms on backward/forwarr transfer on Instance-II. CIFAR suubpoopulation shift) and Task-II. DomainNet (domain shift).
We primarily baseline MOTA against other regularization- based methods (Table 3). aims find a combination parameters that can perform close the multi-task learning strategy, falls shortly behind. By optimizing (i) the number of modes throughout the parameter space against (ii) optimal task allocation per parameter, MOTA can outperform other regularization and replay methods. In particular, MOTA outperforms its component baselines: EWC (where this baseline and MOTA use the EWC regularization term minimizing mode drift), and ensembling (where this baseline and MOTA use the same distance maximization procedure from init to return modes). We show that the optimal combination of these components can yield superior performance.
Though MOTA
Further evaluating MOTA against ensembles, ablation multi-mode strategy where each ensemble mode is sequentially- trained on all tasks without any forgetting strategies, we find that ensembles underperform most baselines and MOTA. Ensembling with independent seeds is almost equivalent in performance to single-task training. Though ensembling maximizes functional diversity across
Table I: Metrics evaluation: We evaluate distance, capacity, and other forgetting measures on PoiitCIIFRR100. Instance-IL (a) presumes the coarse labels are the same between task (5 tasks, 20 labels), thus representative of sub nobpoopultiion shift. Task-IL (b) presumes unique fine labels per task (10 tasks, 10 labels), and is representative of the general continual learning setting. For all comparisons, average task drift begins from parameters updated on Task 1, not from initialization.
(a) Instance-Il.
Methol
Task Driht
Capacity
Rackward Trander
Forward Transfer
Remembering
Forgetting
Sungle Loarning
403
753
413
Louming
(b) Task IL
Method
Arerage Aecurucy
Average Task Deit
Unpaciy
Hockward Transier
Forward Tramster
Remembering
Forgetting
Singl= Yaok Learning
$4.8
T0
23.38850
-247
46.9
75.3
21.7
BWC
21.328.222
8.0
62.7
92.0
MOTA
x10=
22.863.254
69.
Moli-Takk Leaming
23,528.32
Table 2: Varying Datasets: and domain shift dataset (b).
We continue our Task IL evaluation on another task shift dataset (a),
(a) TinylmageNet
Method
Average Accuracy
Backward Transfer
Forward Trausfer
Remembering
Forgetting
Single- lask Learning
65.2
-14.7
65.0
85.3
16.3
EWC
76.7
4.49
74.7
3.45
MOTA
82.7
1,70
81.2
98.3
1.57
Mulu-Tas Learning
90,2
0
0
(b) DomainNet
Method
Average Áccuracy
Backward Transfer
Forward Transter
Remembering
Forgetting
Task Task Learning
34.8
-1.55
22,2
82.5
27.1
FWC
40.2
-982
31.6
90.2
8.57
MOTA
57.5
-1.62
55.9
98.4
2.43
Muski-Task Learning
70.4
0
0
0

---

Under review as a conference paper at ICLR 2023
Table 3: Baseline comparison: Evaluating on Task-IL Split-CIFAR100, we evaluate MOTA against Single/Multi-Tass Learning (the lower/upper bound), regularization (in-line assumptions) and replay parameter isolation (more difficult beat) methods, and ensemble ablations. We keep capacity at most 1x ResNet50 for fair comparison. Baseline configurations are listed in Appendix A.
Method
Average Accuracy
Storage: Model parameters ~
Storage: Replay buffer
Single Task Learning
54.8
23,528,522
EWC (Kirkpatrick ct al.
2017b
66.9
x 23,528,522
Zenke of al. 2017
63.7
x 23,528,522
LWF Holem 2016
61.2
23,528,522
ER (Kiemer el al. 2018
68.2
23,528,522
10,000
A-GEM Chuudhy m
2018b
67.2
x 23,528,522
10,000
La-MAML Gupta e
2020
65.8
23,528,522
10,000
GPM Saha
2024
67.4
23,528,522
10,000
Insembe
2020
62.2
30,116,508
WSN Kang 2024
68.7
23,199,123
MOTA
70.5
2 x 11,1816642
Ensembles (distance max.)
60.1
2 11,181,642
Ensembles (independent sceds)
55.5
11,181,642
Multi-Task Learning
77.1
23,528,522
modes, contains diverse representations per task, and increases the likelihood of mode being closer to multi-task parameter, it does not have a coordinated inference strategy (e.g. weighted mode predictions) nor it capacity-efficien with respect to tasks (updating all modes or subset of modes). Furthermore, ensembles are trained on all tasks and each mode returns an informed probability distribution conditioned on all tasks while MOTA dilutes the joint probability distribution with partially-condiiioonn conditioned probability distributions of updated models with random distributions of non-updated models, thus MOTA would be expected to underperform ensembling.
Unlike prior continual learning methods, we do not assume each parameter must have seen all prior tasks. We introduce efficiency task allocation per parameter. This similarly motivated to a multi-headed architecture (where distinct subnetworks tasks) with a shared header, though we do not need a task index to select a mnsk-specific subnetwork.
explicitly allocated to different subset of
4.2 TRADEOFF BETWEEN ACCURACY AND CAPACITY
In Figure vary the capacity the num- ber of total trainable parameters in proportion to the number of parameters for EWC (us- ing a ResNet152 model). 2x ResNet18 8 is similar in capacity to EWC (ResNet50), and the other architectures of varying number of modes have less than or similar capacity to EWC (ResNet152)..
3xftss S0e
SvReshett9 dodtediue 18
12
2sRmddett 1utesle118
4sRes/ee18
2vResNNt50
We learn from ensemble performance Table 3 that utilizing global geometry alone is not sufficient to improve average accuracy, and that need to optimally allocate tasks per mode. Similarly in Figure 2 we observe a similar inclination for a balance between cap- turing global geometry and optimizing tasks per mode. We would expect that, given a con- stant number of tasks, an increasing number of modes would result improved average accuracy. Instead, the average accuracy gain between 2-6 ResNet1 8s is minimal.
(Reeeet))))))
64
65
0 30
50
13
70
80
30
100
132
120
Figure 2: Varying modes count: Compared to EWC (ResNet50/1522, we evaluate the trade-off between accuracy against capacity (number of modes), given constant number of tasks.
We also observe a trade-off between the number of modes and optimal task allocation per mode. Considering EWC (ResNet50; i.e. 1x ResNet50) and 2 - -33x ResNet50, an increase in the number of modes results in an increase in average accuracy. Considering constant capacity, 4x ResNet18 outperforms 2x ResNet50; however, 3x ResNet50 outperforms 6x ResNet18.
7

---

Under review as a conference paper at ICLR 2023
4.3 CHANGES THE GEOMETRY THE PARAMETER SPACE
From Table 1 the average task drift (drift distance between the next and previous task's parameters) tends to be lower for MOTA than EWC, Single-Task, and Multi-Task Learning. This can be visually observed in the trajectory of the parameters Figure 3
We also observe from Figure that the loss landscape changes drastically between tasks. A region considered to be low-loss by a parameter at task becomes a high-loss region with respect to the next task 1. 1. As each task is added, the sharpness of the basin upon which the EWC parameter exists tends to increase. This change in sharpness tends to be much smaller for the regions in which MOTA modes are located, where the basin still retains similar level of flatness.
Task 1
Task 2
Task
Task 4
Task 5
PO0 198 enu
EWC
POI 90
Mode
Mode
120
100
loss
Task 6
Task
Task 8
Task 9
Task 10
EWC
Mode
20
28
Mode 2
100
Figure 3: Loss Landscape: In-line with Li et al. 2018). we visualize the loss landscape by storing the model parameters along the optimization trajectory per epoch from Task 1-10 (including the last parameter 8*), identify the top two components/directionss 8, n with PCA, and with respect to each task's dataset æt, V/. we plot the loss function L("" + agō + a.,")) with varying interpolation coefficients ass Oky. We plot each set per method across the tasks to show the relative change in flatness/sharpness between tasks. We normalize the loss values all plots jointly between and The trajectory (white line) is the position of the parameter in the parameter space at the f-th task. Note that the loss values are not necessarily synchronized for each parameter between tasks (c.g. the init parameter) the loss for the same parameter may be different for different tasks.
8

---

Under review as a conference paper at ICLR 2023
5 CONCLUSION
Driven by observations in the optimization behavior of multi-task learners, we hypothesize incorpo- rating the broader geometry the parameter space into continual learner for improved adaptation between data distributions. Supported by the formulation of a trade-off between the number of modes and task allocation per mode, we demonstrate that Kode-Optimizer Task Allocation (MOTA) can outperform existing baselines. It can retain a high average accuracy on current and previous data in sub-population, domain, and task shift settings. We also present supporting results on how MOTA influences the sharpness of the loss landscape between tasks, and how accuracy varies with the total capacity of MOTA. With this first step in leveraging the global geometry of loss landscapes changing with tasks, many potential future directions exist. Next steps range from more efficient methods in leveraging the global geometry (c.g. using fewer modes or single network alone, alternative task allocation schemes), demonstrating improved properties with architecture modifications (e.g. improve robustness with changes to regularization terms), to supporting meta learning regimes (e.g. unseen tasks).

---

Under review as a conference paper at ICLR 2023
REFERENCES
Arslan Chaudhry, Puneet Kumar Dokania, Thalaiyasingam Ajanthan, and Philip H. S. Torr. Rie- mannian walk for incremental learning: Understanding forgetting and intransigence. In ECCV, 20.
Arslan Chaudhry, Marc' Aurelio Ranzato, Mareus Rohrbach, and Mohamed Elhoseiny, Efficient lifelong learning with a-gem. ArXiv, abs//812.0044 2018b.
Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K. Dokania, Philip H. S. Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning, 2019. URL
Siddhartha Datta and Nigel Shadbolt, Interpolating compressed parameter subspaces, 2022. URL
petps://arxiv.org/abss
Thang Doan, Seyed Iman Mirzadeh, Joelle Pineau, and Mehrdad Farajtabar. Efficient continual
learning ensembles in neural network subspaces, 2022. URL https:://arxiv.orr//bbs// 09826
Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred A. Hamprecht. Essentially barriers in neural network energy landscape, 2019.
Natalia Díaz- Díaz-Rodríguez, Vincenzo Lomonaco, David Filliat, and Davide Maltoni. Don't forget, there more than forgetting: new metrics for continual learning, 2018. /tKL/https://arcv.oor///
1810. 13166
Stanislav Fort and Stanislaw Jastrzebski. Large scale structure of neural network loss landscapes, 2019.
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry Vetrov, and Andrew Gordon Wilson. Loss surfaces, mode connectivity, and fast ensembling of dnns, 2018.
Gunshi Gupta, Karmesh Yadav, and Liam Paull. Look-ahead meta learning for continual learning. Advances in Neural Information Processing Systems, 33:1 1588-115999 2020.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. arXiv preprint arXiv: 2015.
Haeyong Kang, Rusty John Lloyd Mina, Sultan Rizky Hikmawan Madjid, Jaehong Yoon, Mark Hasegawa-Johnson, Sung Ju Hwang. and Chang D Yoo. Forget-free continual learning with winning subnetworks. In International Conference on Machine Learning, pp. 10734-10750. PMLR, 2022.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kicran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barkkkskaaaiikaa,, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks, 2017a.
James N Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz. Joel Veness, and et. al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences of the United States of America, 1 14 13:3521-3522 2017b.
Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009.
Matthias Lange, Rahaf Aljundi. Mare Masana, Sarah Parisot, Xu Jia, Ale Leonardis, Gregory G. Slabaugh, and Tinne Tuytelaars. Continual learning: A comparative study on how to defy forgetting in classification tasks. ArXiv, abs/1909.08320 2019.
Hao Li, Zheng Xu, Gavin Taylor, Christoph Studer, and Tom Goldstein. Visualizing the loss landscape of neural nets. In Neural Information Processing Systems, 2018.
Zhizhong Li and Derek Hoiem. Learning without forgetting. In ECCV, pp. 614-629, Springer, 2016.
10

---

Under review as a conference paper at ICLR 2023
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning, In Advances in Neural Information Processing Systems. pp. 6467-6476, 2017.
Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7765-7773, 2017.
Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. 1989.
Seyed Iman Mirzadeh, Mchrdad Farajtabar, Dilan Gorur, Razvan Pascanu, and Hassan Ghasemzadeh. Linear mode connectivity in multitask and continual learning, 2020.
Adam Paszke, Sam Gross, Francisco Massa, and al. Pytorch: An imperative style, high- performance deep learning library. In Advances in Neural Information Processing Systems 32, pp. 8024-8035. Curran Associates, Inc., 2019.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation, 2019.
Roger Ratcliff. Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psychological Review, 97(2):285 308, 1990. doi: 10.103770033-2955.99 285.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In CVPR, pp. 2001-2010, 2017.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. arXiv preprint 1rX0v:1810.111 11910, 2018
Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2): 1995. doi: 10. 101880995500995500993188 URL https:://dol.orr/10, 1080/
09540099550639311
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy P Lillicrap, and Greg Wayne. Experience replay for continual learning. arXiv preprint arXiv:1811.1168) 2018.
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. arXiv preprint 2rXiv:2103.0977 097622 2021.
Joan Serrà, Dídac Surís, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention the task. arXiv preprint arXiv:1801.01423,, 2018.
Stanford. Tiny ImageNet Challenge, CS231N Course. URL htRLlitttss//tii--innnnreeeeettr com/
Yeming Wen, Dustin Tran, and Jimmy Ba. Batchensemble: an alternative approach to efficient ensemble and lifelong learning. In International Conference on Learning Representations, 2020. URL
Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in superposition, 2020.
Mitchell Wortsman, Maxwell Horton, Carlos Guestrin, Ali Farhadi, and Mohammad Rastegari. Learning neural network subspaces, 2021.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In Proceedings of the 34th International Conference Machine Learning Volume 70, ICML'17 17, pp. 3987-3995 JMLR.org, 2017.
11

---

Under review as a conference paper at ICLR 2023
A APPENDIX
This appendix is organized as follows:
Appendix
A.1
We provide detail on the experimental configurations.
Appendix A.2
We review the EWC regularization term, given its regular usage as
baseline as well its usage as distance minimization term in MOTA.
Appendix A.3
We provide our complete analysis for the trade-off between the number of modes task allocation.
A.1 EXPERIMENTAL CONFIGURATION
Training: We loaded pre-trained ImageNet weights for each ResNet architecture with PyTorch (Paszke et al. 2019). We trained for 200 epochs, with batch size 512, using A optimizer (learning rate 0.1 with lcycle learning rate policy) train/val/test split of 70/10/20% We train and evaluate (including when averaging the joint probability distribution) using cross- cross-entropy loss function. We used the seed 3407 throughout all operations: for those requiring multiple unique random values (e.g. multiple randomly initialized models), the seed the index of the object (1, 2, ...).
Regularization methods: The regularization strength for weight penalty for EWC and SI is 000 and 100 respectively, with SI dampening term 0.1, and LwF's temperature for distillation loss 2.0.
Replay methods: The memory buffer's budget per class 100. We use 10-7 A-GEM epsilon (parameter to ensure numerical stability of A-GEM). For La-MAML (Gupta et al. 2020 a meta-learning approach with a replay buffer, we retain the per-pane per-parameter learning rate dgg 0.1 and the learning rate of a at 0.3. We use 0.01 learning rate for GPM (Saha
et al. 2021).
Parameter isolation methods: For WSN (Kang et al. 2022). which selectively reuses and expands subnetworks within network, we set layer- wiie wise capacity to 0.5. BatchEnsemble (Wen et al. 2020) uses a base network (slow weights) and stores separate parameters (fast weights) to compute the parameters per ensemble, thus N ensembles do not require N sets parameters. Each ensemble member is responsible for one task (N = 10). We retain -0.5 random sign init for fast weights and 0.5 fast weights learning rate multiplier.
MOTA:
The distance maximization coefficient 9max is 100.0. For the distance minimization procedure in subsequent epochs, we retain the clastic weights consolidation procedure of computing the Fisher information matrix and computing its corresponding regularization term. We retain EWC's lambda Amin = À = 1, 0000
Ensembles methods: We used the same number models in ensemble as the number of modes of the comparable MOTA (N 2 for Table B We retain the same distance maximization coefficients as MOTA, and use unique seeds (1,2....) for each model's random initialization.
A.2 EWC REGULARIZATION TERM
Importance of each parameter is computed for each task by the parameter's corresponding diagonal element from its Fisher Information matrix Given the index of the parameters i (ith element of 8, thh diagonal element of F), importance of the previous task compared to the next task X. can 16--1
compute the EWC regularization term (Kirkpatrick et al. 2017a): D F.(0, - 0,-1)2.
A.3 ANALYSIS ON MULTIPLE MODES VS TASK ALLOCATION TRADE-OFF
First we denote ginit as the initialization parameter, (MIL((.....T) the multi-task parameter trained on tasks 1,... T, and 0i,, as the parameter of mode index i updated on task t.
Lemma Iterating through each task t, for a reference multi-task parameter @MT, the cumulative distance between updated 0z,t and previous 0,t-1 parameter with respect to @MTi will exceed the
12

---

Under review as a conference paper at ICLR 2023
drift between 8, and (
20 ( El=2(01,e 02,1-1)
Proof. By the triangle inequality, the sum of the distances between the previous and the updated parameter with respect to will exceed the drift between the previous and the updated parameter:
(0.. ) (0..--1- - pmm)
(0,. -0,,-1)
(1)
Thus, show that that the cumulative distance between an updated parameter with respect gMT will exceed the drift between the updated and previous parameters. Hence, this cumulative distance also measures the task drift.
EL - -MMIL) Ei=2(0ie - 0s,t-1)
(2)
Definition (Task Allocation) Task Allocation is defined as procedure that allocates of tasks T(i) = (1) to be learnt by a parameter mode Oi,, of index i.
Definition 1.1 (Optimal Task Allocation) Optimal Task Allocation is defined as a procedure that maximizes the number of tasks {t}l allocated per mode of index i, while minimizing the total drift between parameter updates E 2 (0s,. 0i,--1).
Corollary We can approximate Optimal Task Allocation by optimizing the number of tasks K(H allocated to mode against the cumulative distance between an updated Oi. and previous 0,,11 parameter with respect to GMTL (Lemma 1). This results in:
T() ΣΣ (0.. (
T(i) min arg
(3)
RO221
1m1
In other words, T(i) -MMT At least task must be allocated per mode [T(i) 2
Corollary 2 Given L(f (2 y ((o) C(F(07(): ( (()). we use gMTI. (l........))) as the reference multi-tass parameter.
Theorem 1 the number of modes N is optimized against capacity 10) and the set of tasks allocated per mode (T(i) {f)) for N, t E T, then the total task drift is lower in the multi-mode setting than single-mode setting:
N T()
101/N (0l.e.- BINN d=1
M2 ΣT
2 ( (01..d
Proof. Based on Lemma 1, given N modes and optimal task allocation T(i) with respect to the distance between each 0, and gMTI, we can compute the total drift with respect to GMTI, as EN, (0.,, (.TT).
Note that the capacity of an evaluated mode changes between the multi-mode and single-mode setting. We can compute the total drift normalized by capacity (specifically the number of parameter values)
with the squared Euclidean distance averaged by number of dimensions
(MMML
(0
gMT
given [ 10
From Corollary 1, [T(i] is larger when 0, is closer to @T. Thus for threshold T, can decompose the total drift into:
N 2
D
T())
101/N dd1
(fla. MMTL
BIN
TO
TOIT
d=1 (Or.ed 2 + E
T(o) IOIST 10// 10 D (0,,, (0,.d-0M11
(4)
PIN
Consequently, taking the difference total drift (Eqt settings result in the follow trade-off function (Eqt) 5
between multiple-mon against single-mode
We add 1/N to enable comparison between
13

---

Under review as a conference paper at ICLR 2023
single- single-mode and multi- given their N differs.
min E
(
181/N d=
(0s,t.d 02 1 - 1- 02
1
2im1
r())
d=
(Oiied 0 + Σn
T(t) IT
I DIIAA (0
0
r()
| d (01,a,d 0 + It
T(a)
ITIET 1
[o] d 02
2i=1
(5)
7(t) TTIET Notably, for small T (c.g. T= 1). the "
T0 d=11 (0,,,a-0 term only learns a few tasks per mode, lowers the capacity available mode |0|/N, and thus these capacity-inefficiel modes are redundant. Furthermore, as decreases, the functional diversity of a mode is less important, and any random mode can generalize the set of tasks (T) S T. Hence,
TW TOllTMIsT L
ITISST
1 1/1//N (D.,e,d 02 = M 00/N 1-11
T(
ITISST 1 10
1 (O )
(6)
10
If 1, then T O, given:
Σ
T( TO|IT127 1 119//N
10/N d=1 (0,,,,d 1 + Σ
T(t) T ITIIST 1/8//N
19//N 2d=1 (fleed - m
(7)
- (OI,t.d )² + Σ TTIST 00
[o] - (01.e.d 0
Performance would be identical to the single-mode sequential learning case.
If N + 00 (and redundant modes dominate), then 0, given:
T(
TOPT
1O 2 (Os.t.d 0 ) + Σ
T()) ITIIST
10 I (Ole.d - mm
*
T(O) TTIPT
0 1 (01,e, 02 Σ
T(
[0 _
mm?
Though the terms where [T()] > T may reduce the cumulative distance compared to a single- mode setting, extremely large number of modes will result in excess modes only storing one/few tasks. These excess terms will increase, and the cumulative distance from MTL will be greater in the multi-mode setting than the single-mode setting.
0 N oc optimized, then æ < O, given:
N
T(o Tool M
IT
I 10/N (O,,r.d
2 + E
T()) POIIST
1 101// (Die,d oMTT
WAN d=1
Map -=11
T6) TOIST
T -0 (0
m²
199l (01,r.
[o]
For [(()) S T, T.aany any sampled mode will be similarly distant from OMIL thus we can cancel this term both sides.
7() TOITT N²
10 0 -
- mmL < x TOllTOI
p7 (01.a mz
(10)
This result shows that, compared to single-mod mode sequential learning, we optimize the number of modes N, then we can minimize the cumulative distance with respect to gMTL, and thus minimize the total task drift.
In other words, we conclude that optimizing the number of modes N against capacity |0] and tasks allocated per parameter [Ti) can outperform training on a single mode. If we increase N, then we can minimize the total task drift. If N is too large, however, then the number of tasks allocated per parameter [T(i) decreases, and thus increases the number of redundant mode terms (and total task drift).
N.N
('())
10/N 1 d=1
(Br,,d 0.TTI 1 >
1-
-
(0 m
<
(11)
Y 1=1
10 1 MIN 4=1 (0r,.d - M2 < IJ गि (01.t.d M
(()
1 100
(12)
14


Reviews:
[{"review_id": "ciVagSEdfZ", "reviewer": "Reviewer_EVp1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a novel method, Mode-Optimized Task Allocation (MOTA), for continual learning with superior backward and forward transfer performance as well as lower average task drift.\n\nThe paper first identifies the key weakness behind prior regularization-based continual learning methods: they anchor all future parameters to the first task observed, even when doing so degrades transfer performance. MOTA instead aims to leverage the global geometry of the parameter/loss space by instead optimizing $N$ different modes, each having a set of tasks allocated to it. It proves that when $N$ and the set of tasks per mode are optimized against parameter capacity $|\\theta|$, the total task drift is lower than when $N=1$.\n\nIt describes the MOTA algorithm which first initializes the $N$ modes such that the pairwise distance among them are maximized, and updates the parameters for each task via checkpointed gradient updates such that the drift per mode is minimized.\n\nThe paper concludes after presenting empirical comparison of the transfer performance of MOTA with other baseline continual learning methods, which shows that its average accuracy is superior to all except multi-task learning.", "summary_of_the_review": "Overall, the paper clearly identifies a limitation with prior regularization-based continual learning methods and suggests an alternative that is both theoretically and empirically well-supported and convincing. Although parts of its main algorithm lack clear motivation and its conclusion does not offer insights about its potential impact and directions for future research, it is an original and creative work that I strongly recommend.", "strength_and_weaknesses": "Strengths:\n- Provides clear motivation to study multi-mode continual learning (Theorem 1).\n- Shows creativity in the construction of its algorithm.\n- Solid experimental result and its discussion.\n\nWeaknesses:\n- Theoretical justification for the specific steps of its algorithm (Section 3) lacking.\n- In particular, motivations behind (1) the choice of parameter prior that maximizes the pairwise distance and (2) using random-weight-interpolated parameter in its loss function are unclear.\n- The second paragraph of Section 3.2 (starting with \"If a task has a high level of certainty, ...\") is confusingly written and may benefit from an illustrative diagram or a figure.\n- The paper claims that the algorithm does not depend on the knowledge of the task boundaries, and yet the specific implementation of the algorithm seems to depend on it (e.g. regularizing distance with parameter of previous task). Further clarification would be helpful.\n- Conclusion is abrupt and does not provide any direction for future research.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "ZVzRjGWCJgS", "reviewer": "Reviewer_MaED", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new heuristic approach to continual learning in which a set of models (\"modes\") are joinlty updated after each task, as in an ensemble method, but using a novel loss function, which encourages diversity. Some empirical gains (relative to some other heuristic methods) on some task incremental learning image classification benchmarks are shown.", "summary_of_the_review": "The paper proposes some ad-hoc heuristics which are poorly explained and seem lacking in theoretical justification. The experimental results seem inconclusive.", "strength_and_weaknesses": "The idea of using a mixture of sub-models (\"modes\"),, instead of a single large parameter model, seems intuitively reasonable, although is not very novel (there is a large prior literature on ensembles and mixture of experts, etc). Furthermore, the paper makes claims that I think are false, and is very hard to read.\n\nOne of the biggest problems is that the authors claim they do not need to know about task boundaries. Yet it is clear from algorothm 1 that the data is actually provided to the algorithm in chunks, with each chunk corresponding to a different task, and the algorithm is allowed to exploit this fact , by storing different parameter vectors for each mode i and task t, in the form theta(i,t). This seems contradictory. Furthermore, it seems the algorithm stores multiple copies of theta(i,t), one per epoch, to get theta(i,t,e). Thus the running time in step 12 is O(t E N) for each task t, resulting in a total complexity of O(T^2 E N). This seems expensive in time and space - more so than a simple replay buffer strategy (which they do not compare to).\nAnother problem is that their predictions are computed by using an unweighted average of the logits from each mode. What is the justification for this? If the modes are specialized on different tasks, why should they all get to vote equally?\n\nThe presentation is very unclear. For example, what does thm 1 even mean? theta_d^MTL is never defined. And why should we care about how much paramters drift or not? What does this have to do with predictive performance? Also, what assumptions are you making on the data stream (data distribution for each tasks)? \n\nMany terms are either not explained (eg \"EWC\" is not defined), or very poorly explained. For example, the distinction between instance-IL, task-IL and class-IL is very unclear. It seems to have something to do with the label (output) space for each task, Y_t ( some partitioning of a fixed, larger (\"fine grained\") label set. ) . It seems the algorithm assumes ahead of time that every task will always be a C-way classification task, but the \"meaning\" of each label may change across tasks (eg for task 1, 0=dog, 1=cat, 2=horse; for task 2, 0=truck, 1=car, 2=airplane), which we can think of as a hierarchical two-part label, (task, class). So it seems once again that you do need to know the task boundaries.\n\nThe experiments are not very clear. \n- In sec 4.0, What does it mean to say \"We utilize resnets-{18,50,152} with {11, 181, 642 || 23, 528, 522 || 58, 164, 298} adjustable parametrs? What does the || notation mean? What are these adjustable parameters, and which ones are fixed? \n- In sec 4.1, you compare to MTL as your \"gold standard\", but what exactly is this? (I assume you train a single big model with T output heads, each with C classes, in an offline way?) \n- In sec 4.2, the meaning of fig 3 is unclear. It seems that the loss does not vary at all in the subspace of  mode 2, suggesting that a single mode may suffice. Also, for mode 1, the low loss region seems constant across tasks, but this might be an artefact of your regularizaton strategy. Once again, why should we care about how the parameter subspaces change? What does this have to do with predictive performance?\n- In table 3, you only compare to somewhat older baseline methods (<=2018) -are these SOTA? And you only compare on 1 dataset (split cifar-100). Are the differenes between methods significant? (No error bars are reported!) Can the diferences be explained away by hyper-parameter tuning?\n- tables 1-3 are all hard to read (bolding the best and second best might help, and/or focus on fewer metrics).\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "VglFUmBKNx", "reviewer": "Reviewer_x9Gv", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors consider the usage of more than one models for CL problem, and apply two regularization terms to maximize the difference of all models at 1st task and to minimize the parameter drift.\n\nThe paper claims better model quality when compared to other method.", "summary_of_the_review": "Recommend accept, marginally over threshold. Major reasons for not higher score would include the extent for novelty and baseline model selection in the comparison, whether more advanced and newer methods are compared to MOTA.", "strength_and_weaknesses": "Strength\n1) Two regularization are proposed. One to max the difference of all models at 1st task. One to reduce param space drift between two tasks.\n2) The MOTA work should be easy to enable and user friendly for many real world tasks.\n\nQuestion\n1) The two regularization both use the weighted factor when added to the loss. How robust would the model metric be when the weighted factor change?\n2) As the results in Table 3 have base line approach from papers before or in 2018, are they the best base line model for now, or should the comparison add newer approach from papers after or in 2019?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "RjROwoiUvTF", "reviewer": "Reviewer_ydF1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "\nThe paper studies continual learning with multiple modes. The proposed method MOTA is motivated by mode connectivity and ensembles literature. MOTA aims to train $N$ diverse solutions on the first task, while for the next tasks, it uses the average/joint prediction to minimize the training loss and parameter drift. In addition, at the end of training each task, MOTA uses stored checkpoints to further optimize the drift. The authors show the effectiveness of MOTA on various tasks and instance incremental scenarios.\n\n\n", "summary_of_the_review": "Overall, I find the idea of continual learning with multiple modes interesting, I believe the paper needs improvement regarding the organization/presentation. However, I think this is a good first step.\n\n\n\n\n**Update (Post-Rebuttal)**  \nI would like to thank the authors for their response. Given that the authors have addressed most of my concerns and included new baselines, I would like to increase my score.\n\n\n", "strength_and_weaknesses": "### Strengths\n\n\n- The paper is easy to follow, and the method is intuitive. \n\n\n- The idea of continual learning with multiple modes and the trade-off analysis is interesting. \n\n\n\n### Weakness:\n\n\n- First, I should note that in terms of novelty and contributions, continual learning with multiple modes is not new. In fact, in [1] (see Alg. 4, 5), the authors build on the same idea of multiple modes motivated by neural network subspaces [2]. While in this work, there is an additional term to encourage diversity, the authors in [1] report the cosine regularization in [2] does not provide substantial additional benefits. Overall, while I suspect authors may not be aware of [1], in terms of novelty, I believe the core ideas of [1] are similar to this work.\n\n\n\n- The proposed method, MOTA, has some limitations. First, it requires knowledge of task boundaries (to discourage drift). In addition, I believe the experiments do not cover the Class-IL setup. Also, I believe several other related baselines, such as [1] and [3], should have been included in benchmarks.\n\n\n\n- [Minor; did not play a role in my decision] I believe the paper needs additional proofreading. More specifically, there are several typos (e.g., section 1, section 2), and on page 2, the paper refers to a figure on page 8.  \n\n\n\n\n**References**:  \n\n\n[1] Doan, Thang et al. “Efficient Continual Learning Ensembles in Neural Network Subspaces.” ArXiv abs/2202.09826 (2022).\n\n\n[2] Wortsman, Mitchell et al. “Learning Neural Network Subspaces.” ICML 2021. \n\n\n[3] Wen, Yeming et al. “BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning.” ICLR 2020.\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject