Dual Cognitive Architecture: Incorporating Biases and Multi-Memory Systems for Lifelong Learning
Authors: Shruthi Gowda
Bahram Zonooz
Elahe Arani
Venue: Accepted by TMLR ()

Abstract: Artificial neural networks (ANNs) exhibit a narrow scope of expertise on stationary independent data. However, the data in the real world is continuous and dynamic, and ANNs must adapt to novel scenarios while also retaining the learned knowledge to become lifelong learners. The ability of humans to excel at these tasks can be attributed to multiple factors ranging from cognitive computational structures, cognitive biases, and the multi-memory systems in the brain. We incorporate key concepts from each of these to design a novel framework, Dual Cognitive Architecture (DUCA), which includes multiple sub-systems, implicit and explicit knowledge representation dichotomy, inductive bias, and a multi-memory system. DUCA shows improvement across different settings and datasets, and it also exhibits reduced task recency bias, without the need for extra information. To further test the versatility of lifelong learning methods on a challenging distribution shift, we introduce a novel domain-incremental dataset DN4IL. In addition to improving performance on existing benchmarks, DUCA also demonstrates superior performance on this complex dataset.

Published in Transactions on Machine Learning Research (10/2023)
Dual Cognitive Architecture: Incorporating Multi-Memory Systems for Lifelong Learning
Biases and
Shruthi Go Bahram Zonooz 2,3 Elahe Arani
s. gowoda@tue.n b.zononZ@tue.nl L.aroni@gmail.com com
NaaTnfo Europe EEindhoven University of Technology Contribuued equally.
Tom Tom
Reviewed on OpenReview:
https: // opt.reveeerr.......mm net/ forun? id- PByV40h103
Abstract
Artificial neural networks (ANNs) exhibit a narrow scope of expertise on stationary inde- pendent data. However, the data in the real world is continuous and dynamic, and ANNs must adapt to novel scenarios while also retaining the learned knowledge become lifelong learners. The ability of humans to excel at these tasks can be attributed to multiple factors ranging from cognitive computational structures, cognitive biases, and the multi-memory systems in the brain. We incorporate key concepts from each of these to design a novel framework, Dual Cognitive Archilecture (DUCA), which includes multiple sub-systemis, im- plicit and explicit knowledge representation dichotomy, inductive bias, and a multi memory system. The inductive bias learner within DUCA is instrumental encoding shape infor- mation, effectively countering the tendency of ANNS learn local textures. Simultancously, the inclusion of a semantic memory submodule facilitates the gradual consolidation of knowl- edge, replicating the dynamics observed in fast and slow learning systems, reminiscent of the principles underpinning the complementary learning system in human cognition. DUCA shows improvement across different settings and datasets, and it also exhibits reduced task recency bias, without the need for extra information. To further test the versatility of lifelong learning methods on a challenging distribution shift, we introduce a novel domain- incremental dataset DN4IL. In addition improving performance on existing benchmarks, DUCA also demonstrates superior performance this complex dataset.
1 Introduction
Deep learning has seen rapid progress in recent years, and supervised learning agents have achieved superior performance on perception tasks. However, unlike supervised setting, where data is static and indepen- dent and identically distributed, real-world data is changing dynamically. Continual learning (CL) aims to learn multiple tasks when data is streamed sequentially (Parisi et al. 2019). This is crucial in real-world deployment settings, as the model needs to adapt quickly to novel data (plasticity), while also retaining previously learned knowledge (stability). Artificial neural networks (ANN), however, are still not effective lifelong learners, as they often fail to generalize to small changes in distribution and also suffer from forgetting old information when presented with new data rcatastrophic forgetting) (McCloskey & Cohen 1989).
Humans, on the other hand, show better ability to acquire new skills while also retaining previously learned skills to greater extent. This intelligence can be attributed to different factors in human cognition. Multiple theories have been proposed to formulate an overall cognitive architecture, which is a broad domain- generic cognitive computation model that captures the essential structure and process of the mind. Some of those thcories hypothesize that, instcad of a single standalone modulc, multiple modules in the brain share information to excel at a particular task. CLARION (Connectionist learning with rule induction online)
Code public at http::/ggttab..co//leaaaLLLaaaat
2 DL dataset public al. http::::/ggtttbb.coo///////// Lab/DNNIL-Gataseet

---

Published in Transactions on Machine Learning Research (10/2023)
Implicit Module
Explicit Module
Cognitive:
Explicit Learning Knowledge: Direct Source of knowledge:
Knowledge Sharing
Cognitive: Implicit Learning
Knowledge: Indirect
Source of knowledge:
External, Precise
Assimilated, Contextual
Episodic Memory
Inductive Biay Learner
Sensory Data
Working Model
C
Lx
LKSWM
Semantic Memory
Stochastic Momentum
Update
Figure Schematic of Dual Cognilive Archileclure (DUCA). The working model the explicit module learns direct sensory data. Within the implicit module, the inductive bias learner encodes the prior shape knowledge and the semantic memory consolidates information from the explicit module. Only network (semantic memory) is used during inference as it includes consolidated knowledge across all tasks.
(Sun & Franklin 2007 is one such theory that postulates an integrative cognitive architecture, consisting of number of distinct subsystems. It predicates dual representational structure (Chaiken & Trope 1999), where the top level encodes conscious explicit knowledge, while the other encodes indirect implicit information. The two systems interact, share knowledge, and coopcrate to solve tasks. Delving into these underlying architectures and formulating new design can help in the quest to build intelligent agents.
Multiple modules can be instituted instead of a single feedforward network: an explicit module responsible for learning from the standard visual imput and an implicit module that specializes in acquiring and sharing contextual knowledge indirectly. The implicit module can be further divided into more submodules, each providing different information. Inductive biases and semantic memories can as different kinds of implicit knowledge. Inductive biases are pre-stored templates or knowledge that provide some meaningful disposition toward adapting to the continuously evolving world (Chollet. 2019 Theories postulate that after rapidly learning information, gradual consolidation of knowledge transpires in the brain for slow learning of struc- tured information (Kumaran et al. 2016). Thus, the new design incorporates multiple concepts of cognition architectures, the dichotomy of implicit and explicit representations, inductive biases, and multi-memory systems theory.
To this end, we propose Dual Cognitive Architecture (DUCA), multi-module architecture for CL. The explicit working module processes the standard input data. Two different submodules are introduced for the implicit module. The inductive bias learner embeds relevant prior information, and networks are shown to be biased toward textural information (unlike humans that are more biased toward global scmantics) Geirhos el. al. 2019), we propose to utilize global shape information as the prior. Both texture and shape are present in the original image, but ANNs tend to rely more on texture and ignore semantic information. Hence, we utilize the implicit shape information and share it with the explicit module learn more generic and high-level representations. Further, emulate the consolidation of information in the slow-fas multi- memory system, gradual accumulation of knowledge from the explicit. working module is embedded in the
2

---

Published in Transactions on Machine Learning Research (10/2023)
second semantic memory submodule. We show that interacting and leveraging information between these modules can help alleviate catastrophic forgetting, while also increasing the robustness to the distribution shift.
DUCA achieves superior performance across all CL settings on various datasets. DUCA outperforms SOTA CL methods on Ceq-CTFAR10 and Seq-CTFAR100 class-incremental settings. Furthermore, in more real- istic general class-incremental settings where the task boundary is blurry and the classes are not disjoint, DUCA shows significant gains. The addition of inductive bias and semantic memory helps to achieve bet- ter balance between the plasticity-stability trade-off. The prior in the form of shape helps produce generic reprosentations, and this results in DUCA exhibiting reduced task-recency bias. Furthermore, DUCA also shows greater robustness against natural corruption. Finally, to test the capability of CL methods against the distribution shift, we introduce a domain- incremental learning dataset, DN4IL, which is a carefully de- signed subset of the DomainNet dataset (Pong et al. 2019). DUCA shows considerable robustness across all domains on these challenging data, thus establishing the efficacy of our cognitive-inspired CL. architecture. Our contributions are as follows:
Dual Cognitive Architecture (DUCA), novel method that incorporates aspects of cognitive archi- tectures, multi-memory systems, and inductive bias into the CL framework.
Introducing DN4 a challenging dommin-iincremental learning dataset.
Benchmark across different CL settings: class-, task- generalized class,, and domain-incrementi incremental learning.
Analyses on the plasticity-stability trade-off, task recency bias, and robustness to natural corrup- tions.
2 Methodology
2.1 Cognitive Architectures
Cognitive architectures refer to computational models that encapsulate the overall structure of the cognitive process in the brain. The underlying infrastructure of such a model can be leveraged to develop better intelligent systems. Global workspace theory (GWT) (Juliani et al., 2022) postulates that human cognition is composed of a multitude of special-purpose processors and is not a single standalone module. Different sub-modules might encode different contextual information which, when activated, can transfer knowledge to the conscious central workspace to influence and help make better decisions. Furthermore, CLARIÓN (Sun) & Franklin 2007) posits dual-system cognitive architecture with two levels of knowledge representation.
The explicit module encodes direct knowledge that is externally accessible. The implicit module encodes indirect knowledge that is not directly accessible, but can be obtained through some intermediate interpretive or transformational steps. These two modules interact with each other by transferring knowledge between each other.
Inspired by these theories, we formulate method that incorporates some of the key aspects of cognitive architecture into the CL method. A working module, which encodes the direct sensory data, forms the explicit module. A second module that encodes indirect and interpretive information forms the implicit module. The implicit module further includes multiple sub-modules to encode different types of knowledge.
2.2 Inductive Bias
The sub-modules in the implicit module need to encapsulate implicit information that provide more contextual and high-level supervision. One of such knowledge can be prior knowledge inductive bias. Inductive biases are pre-stored templates that cxist implicitly even in carlier stages of the human brain (Pearl & Mackenzie 2018). For instance, cognitive inductive bias may be one of the reasons why humans can focus on the global semantics of objects to make predictions. ANNs, on the other hand, are more prone to rely on local cues and textures (Geirhos et al. 2019). Global semantics or shape information already exists in the visual data, but in an indirect way. The incorporation of shapo-awareness the networks has proven to be more effective approach in acquiring generic representations (Gowda et al. 2022 Hence, we
3

---

Published in Transactions on Machine Learning Research (10/2023)
utilize shape as indirect information in the implicit module. The sub-module uses transformation step to extract the shape and share this inductive bias with the working module. As the standard (RGB) image and its shape counterpart can be viewed as different perspectives/ modialttess of the same data, ensuring that the representation of one modality is consistent with the other increases robustness to spurions correlations that might exist in only one of them.
2.3 Multi Memory System
Moreover, many theories have postulated that an intelligent agent must possess differentially specialized learning memory systems (Kumaran et al. 2016). While one system rapidly learns the individual experience, the other gradually assimilates the knowledge. To emulate this behavior, we establish second sub-module that slowly consolidates the knowledge from the working module.
2.4 Formulation
To this end, we propose novel method Dual Cognitive Architecture (DUCA), which incorporates all these concepts into the CL paradigm. DUCA consists of two modules, the explicit module, and the implicit module. The explicit module has single working model and processes the incoming direct visual data. The implicit module further consists of two submodules, namely the inductive bias learner and the semantic memory. They share relevant contextual information and assimilated knowledge with the explicit module, respectively. Figure T shows the overall architecture.
In the implicit module, semantic memory NsM: consolidates knowledge al stochastic intervals from the work- ing model NwM, in the explicit module. The other submodule, the inductive bias learner NIBL, processes the data and extracts shape information (Section G). Nww processes the RGB data, Nsw consolidates the information from the working module at an update frequency in stochastic manner, and NIBL learns from the shape data. The encoder or the feature extractor network takes an image as the input and produces latent representations, which are then passed to the linear classifier to do object recognition. f represents the combination of the encoder and the classifier, and 6w M. ASM, and 0IBL are the parameters of the three networks.
A CL classification consists of sequence of T' tasks and, during each task, E 1,2..,T samples x. and their corresponding labels ye are drawn from the current task data D.. Furthermore, for each subsequent task, random batch of exemplars is sampled from episodic memory B 2s. An inductive bias (shape) filter is applicd to generate shape samples, Fe, = II:,)) and Ib, = IB(s)). Reservoir sampling (Vitter 1985 is incorporated to replay previous samples. Each of the nelworks Nw and NIBE learns in its own modalily with a supervised cross-entropp loss on both the current samples and the buffer samples:
Lsupwa =CCE(f(xx)
+ (f 0wM). v6)
(1)
LSupIEL =Lan( 0IBL), ve) + LCE(S(E.) :1BL)) yb)
The Knowledge Sharing (KS) objectives are designed to transfer and share information between all modules. KS occurs for current samples and buffered samples. We employ the mean squared error as the objective function for all KS losses. To provide shape supervision to the working model and vice versa, bidirectional decision space similarity constraint (Luks) is enforced to align the features of the two modules.
Luks= =
E (f(2x:011B)) - f(z
(2)
aND.UB
The consolidated structural information in semantic memory is transferred to both the working model and the inductive bias learner by aligning the output space on the buffer samples, which further helps in information retention. The loss functions LKSwM and LKSIBL are as follows;
LKswM = E [m AMM) - f(x;; ww3) =
(3)
LKSI
E [ 0sM) f(zo,; 01BL)
4

---

Published in Transactions on Machine Learning Research (10/2023)
Table 1: Comparison of different methods on standard CL benchmarks (Class-II. Task-IT, and GCTL set-
tings). DUCA shows a consistent improvement over all methods for both buffer sizes.
CIFARIO
SEQ-CIFARIO0
GCIL-CIFARI00
B
МETHОN
CLASS-IL
TASK-IL
CLASS-IL
TASK-IL
UNIFORM
LONGTAIL
JOINT
92.2040.15
98.31
70.6240.64
16.1940.48
60.4541.65
60.10+0.42
SGD
19.622+005
61.0223.33
17.58+0.01
40.46:0.99
10.36++013
9.62+0.21
ER
44.79+1.80
91.19#+0.94
21.40+0.22
61.36++0.3
16.52+0.10
16.20+0.30
+
64.8841.17
91.9241.00
19.60+1.14
62.4941.77
27. 27.7340.98
26.4842.04
Co²L
65.57+1.37
93.43++0.7
31.90+0.38
55.02+00.00
200
ER-ACE
62.081144 92.20110.57
32.4910.95
59.77110.3
27.6410076
25.1012264
CLS-ER
66.19+0.75
93.90+20.0
43.80+1.80
73.49#1.04
35.88+0.41
35.67+0.72
DUCA
70.04+1.07
94.49+0..8
45.38++1.28
76.62#0.16
38.61+0.83
37.11+0.16
ER
57.74.00.7
93.61 10.27
28.02.1..3
68.23
23.62.1066
22.36.1.27
+
72.70+1.36
93.88+0060
41.40+00.0
70.61+0.11
35.80+0.62
34.23+1.19
Co²L
74.26+t0.77
95.90+10.26
39. 2 20.39
62.98
500
ER-ACE
68.4511.78
93.4711100
40.67 10.06
66.4510.71
30.1411.
31.8810.73
CLS-ER
75.22+0.71
94.944+0.53
51.40+1.00 1.00
18.1240.22
38.94+0.
38. 79
DUCA
76.20+0.70
95.95-10.14
54.27
79.80+0.32
43.3410.32
41 41.4440.22
Thus, the overall loss functions for the working model and the inductive bias learner are as follows;
Lw M -Supwn + X(Liss LkSwM)
(4)
LIBL -SupIeL + + CKSIBL)
The semantic memory of the implicit module is updated with a stochastic momentum update (SMU) of the weights of the working model at rate with decay factor of d,
0sm = OsM + (1 (1-d) Owm if ()0,1)<>
More details provided in Algorithm !
We discuss the computational aspect in Section Note that we use semantic memory (0sM) for inference, as it contains consolidated knowledge across all tasks.
3 Experimental Settings
ResNet-18 (He et al. 2016) architecture is used for all experiments. All networks are trained using the SGD optimizer with standard augmentations of random crop and random horizontal flip. The different hyperparameters, tuned per dataset, are provided in E The different CL settings are explained detail in Section D We consider CLass-IT, Domain-IT., and also report the Task-II. settings. Seq-CIFAR10 and al. (2009) for the class-incremental lcarning (Class-IL) settings, which are divided into 5 tasks each. In addition Class-IL, we also consider and evaluate general Class-IL (GCIL) (Mi et al. 2020) the CIFAR100 dataset. For Nomain-incrementa. learning (Domain-IL), we propose novel dataset, DN
Beq-CIFAR100 (Krizhevsky
4 Results
We provide a comparison of our method with standard basclines and multiple other SOTA CL methods. The lower and upper bounds are reported as SGD (standard training) and JOINT (training all tasks to- gether), respectively. We compare with other rehearsal- based methods the literature, namely ER, DER-+ (Buzzoga et al., 2020), Co (Cha et al. 2021). ER-ACE (Caccia al.] (201),, and CLS-ER (Arani et al. 2022). Table shows the average performance in different settings over three seeds. Co utilizes task boundary information, and therefore the GCIL. setting is not applicable. The results taken from the
5

---

Published in Transactions on Machine Learning Research (10/2023)
original works and, if not available, using the original codes, conducted hyperparameter search for the new settings (see Section F. for details).
DUCA achieves the best performance across all datasets in all settings. In the challenging Class-IL setting, we observe a gain of ~50% over DER++, thus showing the efficcyy of adding multiple modules for CL. Furthermore, we report improvements of ~6% on both the Seq- CT and Seq-CTFARION datasets, over CLS-ER, which utilizes two semantic memories in its design. DUCA has single semantic memory, and the additional boost is obtained by prior knowledge from the inductive bias learner. Improvement is prominent even when the memory budget is low (200 buffer size). GCIL represents more realistic setting, the task boundarics are blurry, and classes can reappear and overlap in any task. GCII-Longtail version also introduces an imbalance in the sample distribution. DUCA shows significant improvement on both versions of GIIR-CIFAR1O,, Additional results are provided in Table !
Shape information from the inductive bias learner offers the global high-lovel context, which helps in produc- ing generic representations that are not biased towards learning only the current task at hand. Furthermore, sharing of the knowledge that has been assimilated through the appearance of overlapping classes through the training scheme, further facilities learning in this general setting. The overall results indicate that the dual knowledge sharing between the explicit working module and the implicit inductive bias and semantic memory modules enables both better adaptation to new tasks and information retention.
5 Domain-incremental learning
Intelligent agents deployed in real-world applications need to maintain consistent performance through changes in the data and environment. Domain-IL aims to assess the robustness of the CL methods to the distribution shift. In Domain-IL, the classes in each task remain the same, but the input distribution changes, and this makes for a more plausible use case for evaluation. However, the datasets used in the literature do not fully reflect this setting. For instance, the most common datascts used in the litcrature are different variations (Rotated and Permuted) of the MNIST dataset (LeCun et al. 1998). MNIST is a simple dataset, usually evaluated on MLP networks, and its variations do not reflect the real-world distribution shift challenges that CL method faces (the results for R-MNIST are presented in the Table Cp. Farquhar & Gal (2018) propose fundamental desiderata for CL evaluations and datasets based on real-world use cases. One of the criteria is to possess cross-lask resemblances, which Permuted-MNIST clearly violates. Thus, a different dataset is needed to test the overall capability of a CL method to handle the distributional shift.
5.1 DN4IL Dataset
To this end, we propose II (DomainNel for Domain-II.), which a well-crafted subset of the slandard DomainNet dataset (Peng et al. 2019), used in domain adaptation. DomainNet consists of common objects in six different domains real, clipart, infograph, painting, quickdraw, and sketch. The original DomainNet consists of 59k samples with 345 classes in each domain. The classes have redundancy, and moreover, evaluating the whole dataset can be computationally expensive in a CL. setting. DN4TI, version considers different criteria such as relevance of classes, uniform sample distribution, computational complexity, and case of benchmarking for CL.
All classes were grouped into semantically similar supercategories. Of these, a subset of classes was selected that had relevance to domain shift, while also having maximum overlap with other standard datasets such CIFAR, to facilitate out-of-disttibuttio analyses. 20 supercategories were chosen with 5 classes each (resulting in a total of 100 classes). In addition, to provide balanced dataset, we performed class-wise sampling. First, we sample images per class in cach supercategory and maintain class balance. Second, we choose samples per domain, so that it results in a dataset that has a near-uniform distribution across all classes and domains. The final dataset DNŞIL is succinct, more balanced, and more computationally efficient for benchmarking, thus facilitating rescarch in CL. Furthermore, the new dataset is doemed more plausible for real-world settings and also adheres all evaluation desiderata by (Farquhar & Gal 2018). The challenging distribution shift. between domains provides an apt dataset to test the capability of CL. met
6

---

Published in Transactions on Machine Learning Research (10/2023)
ER
CLS-ER DUCA
70
Plasticity
Stability
50
+ER++
60
40
50
Accuracy
Accuracy
40
30
30
20
20
10
[B|-200
BB/-500
ER
DER++
CLS-ER
DUCA
Figure 2: Accuracy (left) and plasticitystaabiiity analysis (right) on DNŞIL dataset. DUCA substantially
outperforms other methods and with better plasticity-stabiiiy trade-off.
in the Domain-IL sctting. More details, statistics, and visual examples this crafted dataset are provided
in Section H
5.2 DN4IL Performance
Figure 2 (left) reports the results on DN for two different bufler sizes (values are provided in Table 10 DUCA shows considerable performance gain in the average accuracy across all domains, and this can be primarily attributed to the supervision from the shape data. Standard networks tend to exhibit texture bias and learn background or spurious cues (Geirhos et al. 2019) that result in performance degradation when the distribution changes. Learning global shape information of objects, on the other hand, helps in learning generic features that can translate well to other distributions. Semantic memory further helps to consolidate information across domains. Maintaining consistent performance such difficult distribution shifts prove beneficial in real-world applications, and the proficiency of DUCA in this setting can thus open up new avenues for rescarch in cognition-inspired multi-module architectures.
6
Model Analyses
6.1 Cassicity-SStabiliy Trade-off
Plasticity refers to the capability of model to learn new tasks, while stability shows how well it can retain old information. The plasticity-stabiliy, dilemma is long-standing problem in CL, which requires optimal balance between the two. Following SarlfaingSSarfraz et al. (2022), measure each of these to assess the competence of the CL methods. Plasticity is computed as the average performance of each task when first learned (e.g.. the accuracy of the network trained on task T2, evaluated on the test set of Tz). Stability is computed as the average performance of all tasks 1 T-1, after learning the final task T. Figure 2 (right) reports these numbers for the DN dataset. As seen, the ER and DER methods exhibit forgetting and lower stability, and focus only on the newer tasks. CLS-ER shows greater stability, but at the cost of reduced plasticity, However, DUCA shows the highest stability while maintaining comparable plasticity. The shape knowledge helps in learning generic solutions that can translate to new lasks, while the semantic consolidation update at stochastic rates acts a regularization to maintain stable parameter updates. Thus, DUCA strikes a better balance between plasticity and stability.
6.2 Recency-Bias Analysis
Recency bias is a bchavior in which the model predictions tend to be biased toward the current or the most recent task (Wu al. (211)). This is undesirable in CL model, results in a biased solution that forgets the old tasks. To this end, after the end of the training, evaluate the models the test set (of

---

Published in Transactions on Machine Learning Research (10/2023)
Task 1
22.5
Robustness (Corruption)
Task Probablity
0.5
Task
Task:
20.0
0.4
15.0
12.5
0.2
10.0
0.1
7.5
5.0
ER
DER++ CLS-ER
DUCA
1
5
Method
Severity
Figure 3: DUCA shows reduced task recency bias (left), as well as higher robustness against natural corrup-
tion (right) on Seq-CIFAR10 (|B)=200) dataset.
all tasks) and calculate the probability of predicting each task. The output distribution for each test sample is calculated for all classes and the probabilities are averaged per task.
Figure 3 (left)) shows the probabilities for each task on the Seq-CIFAR10 dataset. As shown, the ER and + methods tend incline most of their predictions toward the classes seen in the last task, thus creating a misguided bias. DUCA shows lower bias compared to both of these baselines. CLS-ER exhibits roduced bias due to the presence of multiple memorics, but the distribution is still relatively skewed (with respect to probability of 0.2). DUCA shows more uniform distribution across all tasks. The dual information from the shape data and the consolidated knowledge across tasks helps in breaking away from Occam's s razor pattern of neural networks to default to the easiest solution.
6.3 Robustness
Lifelong agents, when deployed in real-world settings, must be resistant to various factors, such as lighting conditions, weather changes, and other effects of digital imaging, Inconsistency in predictions under different conditions might result in undesirable outcomes, especially safety-critice applications such as autonomous driving. To measure the robustness the CL method against such natural corruptions, we created a dataset by applying fifteen different corruptions, varying levels of severity (1- least severe to 5- most severe corruption).
The performances the fifteen corruptions are averaged at each severity level and are shown in Figure (right). DUCA outperforms all other techniques at all severity levels. ER, DER++, and CLS-ER show a fast. decline in accuracy as severity increases, while DUCA maintains stable performance throughout. Implicit shape information provides a different perspective of the same data to the model, which helps to generale high-level robust representations. DUCA, along with improved continual learning performance, also exhibits improved robustness corruption, thus proving to be a better candidate for deployment real-world applications.
6.4 Task-wise Performance
The average accuracy across all tasks does not provide complete mcasure the ability of a network retain old information while learning new tasks. To better represent the plasticity-taabiliy measure, we report the task- wise performance at the end of each task. After training each task, measure the accuracy on the test set of each of the previous tasks. Figure f reports this for all tasks of DN4IL. The last row represents the performance of each task after the training is completed. ER and DER++ show performance degradation on earlier tasks, as the model continues to train on newer tasks. Both perform better than DUCA on the last task, 71.1 and 68.9 respectively, while DUCA has a performance of 61.1.
However, in continual learning settings, the data arrives continuously and the focus is on both retention of old task performance while performing well the current. task. As seen, the accuracy on the first task (real)

---

Published in Transactions on Machine Learning Research (10/2023)
ER
DER+
CLS-ER
DUCA
After real 59.3
62.3
52.7
50.5
After clip
44.5
64.1
54.2
66.7
57.2
51.5
52.9
61
After igraph
33.8
31.0
26.0
45.7
48.0
27.0
54.4
52.9
23.1
59.0
62.1
26.9
After paint 45.4
37.8
15.9
48.6
5.2
48.4
19.1
48.4
54.99
53.1
23.1
42.6
60.1
60.7
26.6
52.8
After sketch 40.4
43.1
11.8
29.9
58.4
50.0
53.6
16.8
39.9
59.5
55.2
54.7
21.0
42.6
49.8
59.7
63.2
23.7
51.8
61.3
After qdraw
27.6
30.8 8.0
20.2
28.6
71.1
43.5
46.9
13.6
30.6
42.9
68.9
51.1
51.9
18.6
38.1
48.9
52.1
54.9
61.4
19.0
44.5
56.6
61.0
real
clip
igraph
paint
sketch
gdraw
real
clip
igraph
paint
sketch
qdraw
real
clip
igraph paint
sketch
gdraw
real
clip
igraph
paint
sketch qdraw
Figure 4: Tisk-wise performance on DN4IL ((8)- where each task represents domain. DUCA shows more retention of old information without compromising much on current accuracy.
Table 2:
Analyzing the impact of inductive bias and knowledge sharing baselines and DUCA. indicates the use of shape as an augmentation. '-X' indicates the removal of component X. '4' refers to concatenation in the channel dimension.
SEq-CTFAR100
DN4IL
MRTHOD
SPECIFICS
200
500
200
500
ER
ORIGINAL. RGB & SHAPE*
21.40
28.02
26.59
31.01
19.47
23.96
27.45
33.44
DER +
ORIGINAL
29.60
41.40
34.75
41.87
RGB & SHAPE*
24.40
34.30
36.55
40.99
DUCA
ORIGINAL
45.38
54.27
44.23
49.32
-SM (RGB & SHAPE*)
24.34
32.64
36.80
43.88
-SM -IBL (SHAPE ONLY)
18.33
21.98
27.89
31.37
-SM -IBL (RGB-SHAPE)
20.57
25.20
31.52
35.68
-SM -IBL (RGB & SHAPE*)
19.47
23.96
27.45
33.44
IBL (RGB & SHAPE")
42.01
49.55
40.75
43.99
reduces to 27.6 on ER and 43.5 on DER+ after training the six lasks (domains), while the DUCA maintains the accuracy of 54.9. Similarly, after the second task, the performance on the first task decreases (44.5 ER,, 54.2 DER++ 57.2 CLS-ER and 62.9 DUCA), but with DUCA the forgetting is lesser. DUCA reports the highest information retention on older tasks, while also maintaining plasticity. For example, CLS-ER shows better retention of old information, but. at the cost of plasticity. The last task in CLS-ER shows lower performance compared to DUCA (52.1 vs. 61.0). The performance of the current task in DUCA is relatively lesser and can be attributed to the stochastic update rate. Therefore, it's essential to recognize that while the shape inductive bias beneficial to classification task, the observed high performances are a consequence of how the inductive bias is thoughtfully integrated into the proposed architecture within the framework of continual learning, where both stability and plasticity hold equal importance.
To shed more light the performance of each of the modules in DUCA, we also provide the performance of the working model and the inductive bias learner, in Appendix Figure 5 The working model shows better plasticity, while DUCA (semantic memory) displays better stability. Overall, all modules in the proposed approach present unique attributes that improve the learning process and improve performance.
7 Effect of Inductive Bias and Knowledge Sharing
To assess the impact of the specific inductive bias and various ways of integrating into the training franework, we conducted supplementary experiments. In these experiments, we introduced two additional baselines, F.R. and DER++, with the utilization of shape as augmentation technique. Specifically,

---

Published in Transactions on Machine Learning Research (10/2023)
Table 3: Ablation analyze the effect of each component of DUCA on Ceq-C1 and DN
SM IBL
KS (WM++IBL)
SEQ-CIFARI0
DN4IL
/
70.04+1007
44.23
x
69. 191228+1.11
40. 30.35#1.31
69.21+1.46
76.
X
/
61.61:1..2
37.33
X
14.7911.86
26.59110.1
included the Sobel filter in the augmentation list, alongside RandomCrop and RandomHorizontalFiii, and proceeded with continual training. The results presented in Table 2 demonstrate that this approach yields inferior performance compared to the baseline models trained solely on RGB images. On DN4IL dataset, the performance is slightly better than baseline, shape is more discriminative and important feature in this dataset. Thus the incorporation of shape as augmentation strategy appears to yield suboptimal feature representations, and is also dependent on the dataset.
We also conduct various ablations on the DUCA framework. Specifically, we perform isolations and exclusions of different clements within DUCA, including IBL and SM. Instead, we subject the base network (DUCA -SM -IBI.) three distinct training conditions: (1) exclusive training on shape images (Shape only), (2) concurrent training on both RGB and shape images
and (3) training on RGB images with the augmentation (RGB & Shape*). It is worth noting that shape information contributes valuable global semantic insights that complement the visually rich RGB data, thus emphasizing the necessity of both modalities for achieving enhanced generalization. However, training a single network on both distributions simultaneously may not always yield optimal utilization of this information.
incorporation of shape filter
Finally, we train the base model within the framework by excluding one component at a time, namely SM and IBL. We train the working model with but excluding the IBI, while also introducing shape as augmentation. The results (-IBL) show improvement due to the presence of the semantic memory module. Nevertheless, the best performance is achieved when incorporating shape information through an alternative supervisory network, namely the IBL.
These experiments underscore the critical importance of the specific method used to induce this knowledge, highlighting its pivotal role in enhancing the overall training process. In our pursuit bridge the distribution gap between RGB and shape images, we have leveraged knowledge distillation as a means of self-teaching, where distinct networks collaboratively engage in mutual lcarning and knowledge sharing. This approach not only sheds light on the significance of effective knowledge sharing but also offers a promising avenue for improving model performance and generalization in complex (visual) tasks.
8 Ablation Study
DUCA architecture comprises multiple components, each contributing to the efficacy of the method. The explicit module has the working model, and the implicit module has semantic memory (SM) and inductive bias learner (IBL). Disentangling different components in the DUCA can provide more insight into the contribution of each of them the overall performance.
Table 3 reports the ablation study with respect to each of these components on both the CIFARI0 and DN,IL datasets. Considering the more complex DN1IL dataset, the ER accuracy without any of our components is 26.59. Adding cognitive bias (IBL) improves performance by 40%. Shape information plays prominent rolc, as networks need to learn the global semantics of the objects, rather than background or spurious textural information, to translate performance across domains. Adding the dual-memory component (SM) shows an increase of approximately 49% over the vanilla baseline. Furthermore, the KS between explicit and implicit modules on current experiences also plays key role in performance gain. Combining both of these cognitive components and, in general, following the modtie design shows gain of 66%. A similar trend is seen on Seq-CIFAR10.
10

---

Published in Transactions on Machine Learning Research (10/2023)
9
Related Works
Rehearsal-based approaches, which revisit examples from the past alleviate catastrophic lorgetting, have been effective in challenging CL scenarios (Farquhar & Gal 2018). Experience Replay (ER) (Riemer et al. 2018 methods use episodic memory to retain previously seen samples for replay purposes. DER++ (Buzzega et al. 2020) adds consistency loss on logits, in addition to the ER strategy. In situations where memory limitations impose constraints on buffer size, such as in edge devices, it has been observed that rehearsal- based methods are susceptible to overfitting on the data stored in the buffer (Bhat et al. 2022). To address this, CO CCha et al., 2021 uses contrastive learning from the self-supervised learning domain to generate transferable representations. ER-ACE (Caccia et al., 2021) targets the representation drift problem in online CL and develops a technique to use separate losses for current and buffer samples. All of these methods limit the architecture to single stand-alone network, contrary to the biological workings of the brain.
CLS-ER (Arani et al. 2022 proposed multi-network approach that emulates fast and slow learning systems by using two semantic memories, each aggregating weights at different times. Though CLSSER utilizes the multi-memory design, sharing of different kinds of knowledge is not leveraged, and hence presents method with limited scope. DUCA digresses from the standard architectures and proposes a multi-module design that is inspired by cognitive computational architectures. incorporates multiple submodules, each sharing different knowledge develop an effective continual learner that has better gencralization and robustness.
10 Conclusion
We introduced novel framework for continual learning that incorporates concepts inspired by cognitive ar- chitectures, high-level cognitive biases, and the multi-memory system. Dual Cognilive Archilecture (DUCA), includes multiple subsystems with dual knowledge representation. DUCA designed a dichotomy of explicit and implicit modules in which information is selected, maintained, and shared with each other to enable better generalization and robustness. DUCA outperformed on Seq-CIFAR10 and Seq-CIFAR100 on the Class-IL. setting. In addition, it also showed a significant gain in the more realistic and challenging GCIL, setting. Through different analyses, we showed a better plasticity-stability balance.
Furthermore, shape prior and knowledge consolidation helps to learn more generic solutions, indicated by the reduced problem of task recency bias and greater robustness against natural corruptions. Furthermore, we introdeeed a challenging domain-IL dataset, DN4IL, with six disparate domains. The significant im- provement of DUCA on this complex distribution shift demonstrates the benefits of shape context, which helps the network to converge on a generic solution, rather than a simple texture- bexture-biased one. The objective of this work was to develop a framework that incorporates elements of cognitive architecture mitigate the forgetting problem and enhance generalization and robustness.
11 Future Work
Here, we delve into the potential for extending our current research, acknowledging its applicability to a diverse range of modalities. The adaptability of DUCA serves as a robust foundation for further exploration. As we broaden the scope of DUCA beyond the image domain, an essential consideration is the identification of pertinent inductive biases tailored to the specific modality in question. For example, when venturing into the audio domain, a promising avenue involves the utilization of spectrogram representations. These representations effectively convert. audio waveforms into visual data, encompassing both frequency and time- domain information. The integration of phonemes, the fundamental units of spoken language, holds the potential to enhance DUCA's 's effectiveness in tasks such as speech understanding, speaker identification, and language processing.
The collaboration between the core DUCA architecture and modality-specifff inductive biases creates a syn- ergy that drives knowledge sharing and learning capabilities. This collaborative architecture yields more generic and robust representations, substantially enhancing overall performance. Furthermore, the gradual accumulation of semantic memory emerges as a valuable asset, particularly in scenarios involving the contin-
11

---

Published in Transactions on Machine Learning Research (10/2023)
nous infiux of data from various modalities. It mitigates the risk of forgetting and empowers the framework to maintain its adaptability over time.
These modality-specifie adaptations, guided by the intrinsic principles and mechanisms of DUCA, open the door to exciting future directions. They offer the potential to advance lifelong learning and adaptability in a multitude of domains. We anticipate that our preliminary work will serve as a cornerstone for future research endeavors, including investigations into various cognitive biases and more efficient design methodologies. Ultimately, we hope pave the way for the advancement of lifelong learning methods for ANNs.
Acknowledgments
The research was conducted when all the authors were affiliated with Advanced Research Lab, NavInfo Europe, The Netherlands.
References
Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general continual learning method based complementary learning system. In International Conference Learning Represenla- tions, 2022. URL https:///ppnnrevie: net//oorurrid-xxxx/VWEEEYY
Prashant Shivaram Bhat, Bahram Zonooz, and Flahe Arani. Consistency the key to further mitigating catastrophic forgetting in continual learning. In Conference on Lifelong Learning Agents, pp. 1195 1212. PMLR, 2022.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: strong, simple baseline. Advonces in neural information processing systems, 2020.
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. In International Conference on Learning Representations, 2021.
Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co2l: Contrastive continual learning. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9516-9525, 2021.
Shelly Chaiken and Yaacov Trope. Dual-process theories in social psychology. Guilford Press, 1999.
François Chollet. On the measure of intelligence. arXiv preprint arXiv:1911.01547 2019.
Lijun Ding and Ardeshir Goshtasby. On the canny edge detector. Pattern Recognition, 34(3):721-725, 2001.
Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning. arXiv preprint arXiv:1805.097 09733, 2018
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel. Imagenet-trained CNNs are biased Lowards texture; increasing shape bias improves accuracy and robustncss. International Conference on Learning Representations, 2019. URL net//oruumiid-Byyhhooooe
Shruthi Gowda, Bahram Zonooz, and Elahe Arani. Inbiased: Inductive bias distillation to improve gen- cralization and robustness through shape-awareness. In Conference on Lifelong Learning Agents, pp. 1042-1042. PMLR, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEFE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Arthur Juliani, Kai Arulkumaran, Shuntaro Sasai, and Ryota Kanai. On the link between conscious function and general intelligence in humans and machines. arXiv preprint orXi::2044.051 051333 2022.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
12

---

Published in Transactions on Machine Learning Research (10/2023)
Dharshan Kumaran, Demis Hassabis, and James L McClelland. What learning systems do intelligent agents need? complementary learning systems theory updated. Trends in cognilive sciences, 20(7):512 534, 2016.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-basee learning applied docu- 22222,,,, 1998.
ment. recognition. Proceedings of the IEFE, 86(11):2278 2324,
Michael McCloskey and Neal Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109 165. Elsevier, 1989.
Fei Mi, Lingjing Kong, Tao Lin, Kaicheng Yu, and Boi Faltings. Generalized class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 240 241, 2020.
German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Newral Nelworks, 113:54-71. 71, 2019.
Judea Pearl and Dana Mackenzie. The book of why: the new science of cuuse and effect. Basic books, 2018.
Xingchao Peng, Qinxun Bai, Xide Xia, Zijum Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVP international conference on compuler vision, pp. 1406 1415, 2019.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference. In International Conference on Learning Represenlations, 2018.
Fahad Sarfraz, Elahe Arani, and Bahram Zonooz. Synergy between synaptic consolidation and experience replay for general continual learning. In Conference Lifelong Learning Agents, pp. 920-936 PMLR, 2022.
Irwin Sobel and Gary Feldman. A 3x3 isotropic gradient operator for image processing. a talk at the Stanford Artificial Project in, pp. 271-272, 1968.
taxonomy and some examples., Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. arxiv preprint arXiv:1904.0770 2019.
Ron Sun and Stan Franklin. Computational models of consciousness: 2007.
Jeffrey S Vitter. Random sampling with reservoir. ACM Transactions on Mathematical Software (TOMS), 11(1):37-57, 1985.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. Tn Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pp. 374 382, 2019.
13

---

Published in Transactions on Machine Learning Research (10/2023)
A DUCA Algorithm
Algorithm Dual Cognitive Architecture (DUCA)
1: Input: Datasct D,, Buffer B
2: Initialize: Three nelworks: Encoder and classifier / parameterized by Own M. Osas, and 1IBL
for all tasks t E 1, do
Sample mini-batch: (Iez Dr
Extract shape images: Ic. (Bx)) where IB is Sobel filter
6: Ssupwar = (C ww)):ww)))) He))
7:
LSupIBL = LCE (f (1
8:
if B 0 then
9:
Sample mini- batch: (xb, yb) ~ B
10:
Extract shape images: Tb, = (x+b)
11:
Calculate the supervised loss:
12:
LSupwM += (Jrs: 0w yh)
13:
CsupraL += Lo (.b,;01BL) 0IBL), y6)
14:
Knowledge sharing from semantic memory to working model and inductive bias learner:
15:
LKSWM = ||] I (an; 0sM) - (a,; Ow))22
16:
CKSIBL = ( Osss) - f(zo; 11
17:
Bidirectional knowledge sharing between working model and inductive bias learner:
18:
LakS E [1; (w))) (1x 01
D~D,
19:
Calculate total loss:
20:
LWM = Lsupw " + X(Coik + LKSw))
21:
LIBL= = LSupraL + 7Lonks + LKSIaL)
22:
Update both working model and inductive bias learner: OwM, OIBL Stochastically update semantic memory:
23:
24:
Sample (O, 1);
25:
if
then
26:
0sM = d- Osm + (1 Owm
27:
Update memory buffer
28: Return: model ASM
B Effect of Task Sequence
Figure presents the task-wise performance of all the three networks in the DUCA architecture, on DNAIL dataset. Semantic memory helps to retain information by maintaining high accuracy older tasks and is more stable. The performance of the current task is relatively lower than that of the working model and could be due to the stochastic update rate of this model. The working model has better performance on new tasks and is more plastic. Inductive bias leaner is evaluated on the transformed data (shape) and also achieves a balance between plasticity and stability. In general, all modules in our proposed method present unique attributes that improve the learning process by improving performance and reducing catastrophic forgetting. Table reports the results of CIFAR100 on different numbers of tasks. Even when the number of tasks increases, our method consistently improves over the baselines.
Table 4: Comparison Ceq- dataset. for different number of tasks 500 buller size.
METHOD
5-TASKS
10-TASKS
20- Te-TASKS
ER
28.0240.31
21.4910.47
16.5210.86
DER++
41.40+0096
36.20+0.52
22.25+5.87
DUCA
53.23=1.62
41.09-0.12
33.60+0.25
14

---

Published in Transactions on Machine Learning Research (10/2023)
Semantic Memory (Default)
Working Model
Inductive Blas Learner
After T1
60.5
66.5
54.5
After T2
61.4
61.4
57.0
71.2
46.5
65.4
59.0
50.8
-39.7
22.3
After T4
60.1
60.7
26.6
52.8
55.2
54.8
21.5
55.5
42.0
44.8
15.4
40.3
After T5 55.59.7
63.2
23.7
61.
54.3
60.8
20.1
45.6
64.7
44.6
53.1
15.1
36.6
61.9
56.6
61.0
47.3
54.2
36.6
47.7
71.4
38.2
48
11.5
29.0
71.2
T1
T2
T3
T4
T5
T6
T1
T2
13
T4
T5
T6
T1
T2
T3
T4
T5
T6
Figure 5: Task probability analysis of all DUCA components DN dataset with 500 buffer size. Semantic memory displays better stability while the working model displays better plasticity.
Table 5: Comparison on R-MNIST dataset with different buffer sizes.
METHOD
200
500
ER
85.01
$8.9111.44
DER-4
90.43=1.87
92.77
CLS-ER
92.2610018
94.0610.07
DUCA
93.53#+021
94.98+0004
C Evaluation on R-MNIST
The Rotated MNIST (R-MNIST) dataset is a variant the MNIST datasct, which is widely used for domain- incremental learning in continual learning. In the R-MNIST dataset, the digits are rotated by fixed degrees. This rotation introduces variations in the orientation of the digits, making machine learning algorithms to accurately classify them. It tests the robustness and adaptability of models, as they need to recognize and classify the digits regardless of their orientation. The results, as presented in Table indicate that DUCA performs better than of her methods in this domain--incremental setting.
more challenging task for
D Setting and Datasets
We evaluate all methods in different CL settings. Van de Ven & Tolias (2019) describes three different settings based on increasing difficulty: task-incremental learning (Task-IL), iamain-inerrmmntall learning (Domain-IT.), and class-incremental learning (Class-IT.). In Class-II. each new task consists of novel classes, and the network must learn both new classes while retaining information about the old ones. Task-IL is similar to Class-IL but assumes that task labels are accessible in both training and inference. In Domain-IL, the classes remain the same for each task, but the distribution varies for each task. We report the results for all three settings on the relevant datasets. CLass-IT. is the most complex setting of the three and widely studied. However, contemporary research tends adopt simplifications when exploring CLass-Il setting, such the assumption of the same number of classes across different tasks, the absence of reappearance of classes, and the sample distribution per class being well balanced. In Generalized Class-IL IL (GCIL) Mi et al. (2020), the number of classes in each task is not fixed, and the classes can reappear with varying sample sizes. GCIL samples the number of classes and samples from a probabilistic distribution. The two variations are Uniform (fixed uniform sample distribution over all classes) and Longtail (with class imbalance). We report results on all three settings: Task-II, Domain-IL,, and CLass-IL. Furthermore, we also consider the GCIL setting for one of the datasets as an additional evaluation setting. All reported results are averaged three random seeds.
15

---

Published in Transactions on Machine Learning Research (10/2023)
Table 6: Search ranges for tuning hyperparameters.
METHOD
HYPERPARAM
SEARCH RANGE
METHOD
HYPERPARAM
SEARCH RANGE
ER
Ir
[0.01, 0.03, 0.1, 0.5)
Ir
[0.01, 0.03, 0.1]
lr
[0.01, 0.03, 0.1]
[0.1, 0.2, 0.3]
+
[0.1, 0.2, 0.5]
[0:1:0.1]
CLS-ER
3
[0.5, 1.0]
rs
[0:1:0.1]
Ir
[0.01, 0.03, 0.1]
[0.99,0.999
[0.01, 0.1, 0.5]
[0.99, 0.999)
CO
k
[0.2, 0.5] [0.01, 0.05]
tr
[0.01, 0.03, 0.1]
[0:10.11
[100, 150]
DUCA
[0.01, 0.1]
ER-ACE
lr
[0.01, 0.03, 0.1, 0.5]
7
[0.01, 0.1]
E
Hyperparameters
For the settings and datasets for which the results are not available in the original papers, using their original codes, we conducted hyperparameter search for each of the new settings. To this end, we utilize small validation set from the training set to tune the hyperparameters. We adopted this tuning scheme because it aligns with the approach employed in the baseline methods with which we conducted comparisons (Buzzega
ct al. 2020
Cha et al.
2021 Caccia et al. 2021 Arani et al. 2022
For Seq- CIFAR10, the results are taken from the original articles (Buzzega et al., 2020 Cha et al. 2021
Caccia et al. 2021 Arani et al. 2022).. For the other datasets (and for each buffer size), we ran a grid search over the hyperparamcters reported in the paper. For SOq-CIFAR100 and CCIL-CIFAR100, we based the search range using Seq-CIFAR10 hyperparameters
reference point. The search ranges are reported in Table [6] Once we find the right hyperparameter, we train the whole training set and report the test set accuracy.
DN4IL dataset is more complex compared to the CIFAR versions and includes images of larger sizes. Hence, we consider the Soq--TnnTmmagen tuning. The learning rate Ir, the number of epochs, and the batch size are similar across the datasets. The ema update rate is lower for more complex datasets, as shown in CLS-ER. [0.01,0.1] with step size of 0.02 for CLS-ER and DUCA. The different hyperparameters chosen for the baselines, after tuning, are reported in Table
hyperparameters the respective paper as a reference point for further
chosen in the range of
The different hyperparameters chosen for DUCA are shown in Table S Hyperparameters: Ir, batch size, number of epochs and decay factor are uniform across all datasets. The stochastic update rate similar to those of CLS-ER. The hyperparameters are stable across settings and datascts and also complement cach other. The loss balancing weights (A and y respectively) are also similar across all datasets. Therefore, DUCA does not require extensive fine-tuning across different datasets and settings.
F Model Complexity
We discuss the computational complexity aspoct our proposed method. DUCA involves three networks during training; however, in inference, only one network used (SM module). Therefore, for inference purposes, the MAC count, the number of parameters, and the computational capacity remain the same in the other single-network methods.
The training cost requires three forward passes, as it consists of three different modules. ER, DER++, CO and ER-ACE have a single network. CLS-ER ER also has three networks and therefore requires forward passes, DUCA has training complexity similar that of CLS-ER; however, outperforms CI.S-ER in all provided metrics. On the memory front, similar all methods, we save memory samples based on the memory budget allotted (200 and 500 in the experiments). There additional memory requirements, as do not save
16

---

Published in Transactions on Machine Learning Research (10/2023)
Table 7: Selected hyperparameters for all baselines.
DATASET
[B
METHOD
HYPERPARAMETERS
ER
1==0.1
DER++
1r=0.03, a=0.1, 8=0.5
200
COºL
Ir:0.5, r:0.5, x:0.2, x*:0.01, e:100
ER-ACE
1r=0.01
CLS-ER
0.1 -= r 0.10.1, =0=0. ay ap=0.999, 0.999
ER
1r=0.1
+
1r=0.03, a=0.1, 8=0.5
500
CO
Ir:0.5, r:0.5, k:0.2, k*:0.01, e:100
ER-ACE
1--0.01
CLS-ER
1r=0.1 x=0.15, rp=0.1, 0000. 0p 0p-9, a, =0.999
ER
(r=0.1
+
Ir 0.03, a=0.5, 8-0.1
200
ER-ACE
0.1
CLS-ER
0- 0= 0,7. 0.6. 0p-9.999, 0,99.999
GCIL-CIFAR100
ER
0-1
DER++
Ir 0.03, a=0.2, 8-0.1
500
ER-ACE
1r=0.1
CLS-ER
0-1 2-0.1, rp 0.7, r.=0.6, 0p-9.999, 0,99.9
ER
1r=0.1
200
DER+
Ir 0.00.3,, a=0.1, 8=1.0 0.05 r, r.=0.04, op-0.999, 0, 0,9999
CLS-ER
DN4IL
ER
0.1
500
DER+
Ir=0.03, a=0.5, 8=0.1 (r=0.05 1=0.1, 10 r.=0.05, a,=0,999, a, 00.999
CLS-ER
Table Selected hyperparameters for DUCA across different settings. The learning rate set. 0.03, batch
size 32, and epochs to 50 respectively for all the datasets. The decay factor d is always set 0.999.
DATASET
[
X
7
DATASET
[
7
SEQ-CIFAR10
200
0.2
0.1
0.1
GCIL-CIFAR100
200
0.09
0.1
0.01
500
0.2
0.1
0.1
500
0.09
0.1
0.01
SRQ-CTFAR100
200
0.1
0.1
0.01
DN4IT.
200
0.06
0.1
0.01
500
0.06
0.1
0.01
500
0.08
0.1
0.01
any extra information (such logits in DER++) to be used later in our objectives. Additionally, there are additional data requirements. The number of parameters is reported in Table
We considered the human brain, cognition, as the most intelligent agent and wanted to incorporate some of the underlying workings into the neural network architecture. Therefore, our goal was to design framework inspired by elements of cognitive architecture that reduce forgetting while exhibiting better generalization and robustness. Our work a preliminary attempt to incorporate elements based on cognitive architecture into the CL algorithm gauge the gain in reducing forgetting. We hope the promising results result future work that explores different kinds of cognitive biases and interactions and moves towards more efficient designs.
17

---

Published in Transactions on Machine Learning Research (10/2023)
Table 9: Comparison of the number of parameters for different methods using ResNet18 architecture and
trained CeF dataset.
METHOD
TRAINING
INFERENCE
ER
11
+ER++
11
CO
23
11
ER-ACE
11
11
CLS-ER
33
11
DUCA
33
11
Algorithm 2 Sobel Algorithm - Shape Extraction
Up-sample the images to twice the original size: Trgb (gggg)
Reduce noisy edges: Ig Gaussian Blur (Ergb kernel size = 3)
--1 0
+1
-1
-2
3: Get Sobel kernels: S² = -2 0
+2
and Sg =
0
0
0
-1
+1
+1
+2
+1
4: Apply Sobel kernels: Idz = Ig * Sz and Fdy = Fg * Sy (* is 2-D convolution operation)
5: The edge magnitude: Ishape = dd + +
Down-sample to original image size: Tshape ds((Tshope)
G Inductive Bias
The shape extraction is performed by applying a filter on the input image. Multiple filters (such as Canny DDing & Goshtasby, 2201)) Prewitt) were considered, but the Sobel filter (Sobel & Feldman 1968) chosen because it produces a more realistic output by being precise and also smoothing the edges (Gowda 2022 sec Algorithm 2 Figure f shows few examples of applying the Sobel operator on the original ct al.
RGB images. The Sobel output is fed to the IBL model.
The DUCA framework can be extended to other domains by selecting appropriate inductive biases relevant to those domains. For example, in the audio domain, one viable option involves leveraging spectrogram representations, which transform audio waveforms into visual data capturing both frequency and time- domain information. Alternatively, the incorporation of phonemes, representing the fundamental sound units in spoken language and carrying essential linguistic information, for speech understanding, speaker identification, and language processing. Additionally, the extraction of pitch and timbre features proves valuable for tasks like melody extraction, intonation analysis, and various music-rclated applications, providing insights into the acoustic characteristics of audio signals.
enhance the framework's capabilities
DN4IL Dataset
We introduce a new dataset for the Domain-IL setting. It is a subset of the standard DomainNet dataset (Peng et al.) 2019) used in domain adaptation. It consists of six different domains: real, clipart, infograph, painting, quickdraw, and sketch. The shift in distribution between domains challenging. A few examples can be seen in Figure
Each domain includes 345 classes, and the overall dataset consists of ~59000 samples. The classes have redundancy, and also evaluating on the whole datasct can be computationally expensive for CL settings. Therefore, we create subset by grouping semantically similar classes into 20 supercategories (considering the class overlap between other standard datasets can also facilitate OOD analysis). Fach super category has five classes each, which results in a total 100 classes. The specifications of the classes are given in Table [1 The datasct consists of 67080 training images and 19464 test images. The image size for all exporiments is chosen as 64x64 (the normalize transform not applied in augmentations).
18

---

Published in Transactions on Machine Learning Research (10/2023)
Input
Sobel
Figure 6: Visual examples of the shape images using Sobel operator
Table 10: Accuracy on the proposed DN4IL dataset for the Domain-Il. setting. DUCA shows significant improvement in all disparate and challenging domains.
B
METHОN
REAL
CLIPART
INFOCRAPH
PAINTING
SKETCH
QUICKDRAW
Acc
JOINT
59.93:1.00
SGD
9.98
19.97+0.31
2.32+:020
6.58+0.34
14.91
01.23:0.11
20.83:10.21
ER
20.08+00455
26.37+00.35
5.56+0.39
13.9240.91
23.69#1.54
69.95=0.56
26. 26.59+0.31
DER+
33.6611.65
37.24:0.64
9.80+10.3
24.16:11.17
34.37+2.00
09726
34.75:10.87
200
CLS-ER
45.53110.8
49.17:1.12
15.79=0.48
35.80+
48.03
1..4
40.8311.07
DUCA
47.5240.22
54.69-10.10
15.70=0.33
37.5440.30
51.98.10.66
58.80=0.18
44.23+0005
27.5440.05
31.89++0.93
7.89+0045
19.39:+1.02
28.36+1.35
70.96
31.0110.62
DER+ 44.49
46.17-4..3
14.01-0.23
33.44+0.90
43.59+1.11
69.53:0.29
41.8740.63
500
CLS-ER
19.85=0.88
51.4140.34
18.17=0.08
37.9440.91
49.02+1.57
55.63=0071
43.41+0.50
DUCA
54.77:10.15
60.37:40.75
19.35=0.39
44.50+t0.43
56.3440.53
60.61+1.73
49.32++.23
real
clipart
infograph
painting
sketch
quickdraw
airplane
flaoorrs
Ice cream
Figure 7: Visual examples of DNATI. dataset
19

---

Published in Transactions on Machine Learning Research (10/2023)
10000
3000
Samples
8000
Campe
6000
2000
4000
1500
1000
2000
500
resl
clipart
infogragh
painting paintig
Sketch
quickdraw
0
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18 19
20
Domains
Super Categories
Figure 8: Number of samples per domain and per supercategory in DNAIL dataset.
Table 11: Details on supercategory and classes in DNAIL dataset.
SCPERCATEGORY
CLASS
1
SMALL ANIMALS
MOUSE TIGER
SQUIRREL
RABBET
DOG
RACOGION
MEDIUM ANIMALS
DEAR
LION
PANDA
ZEBRA
ANIMALS
HORSE
KANGAROO
COW
OCTOPUS
5
NON-INSECT INVERTEDRNTES
SNAIL
SCORPION
SPIDER
LOIST
CRAB
6
INSECTS
BGE
BUTTERFLY
MOSQUITO
BIRD
BAT
7 VEHICLE
BUS
BICYCLE
MOTORBIKE
TRAIN
PICKUP TRUCK
SKY-VEHICL
AIRPLANE
LYVING_SAUCO
AIRCRAPT CARRIEE
DELICOPTER
BOT_AIR AIR BALLOON
STRAWDERRY
PEAR
APPLE
WATERMELON
VEGETABLES
CARBOT
ASPARAGUS
MUSHROOM
ONION
BROCCOLI
11
MUSIC
TROMBONE
VIOLIN
CELLO
GUTTAR
CLARINET
12 PURNITURE
CHAIR
DRESSER
TADLE
COUCIL
DED
HOUSEHOLD ELECTRICAL DEVICES
CLOCK
FLOOR LAMP
TELEPHONE
TELEVISION
KEYBOARD
SAW
HAMMER
SCREWDRIVER
SCISSORS
&
MAN-MADE OUTDOOR
SKYSCRAPER
WINDMILL
HOUSE
CASTLE
BRIDGE
17
NATURE
CLOUD
BUSH
OCEAN
RIVER
MOUNTAIN
FOOD
BIRTHDAY CAKE
HAMBURGER
ICE CREAM
SANDWICH
PIZZA
STATIONARY
CALENDAR
MARKER
MAP
ERASER
PENCIL
HOUSEHOLD ITEMS
PAN
WINE
1000
Samples
600
200
500
ractio
240/
whal
3hath
soiphi
sco
1ob3
Cras
bassuu
moto
Tyin 28 uter
hot, all MEN
water rerii 22000
dr933bl
co
538.
pickup
has
Classes
Figure 9: Number of overall samples per class in DNAIL dataset.
20

---

Published in Transactions on Machine Learning Research (10/2023)
real
painting
Samples
500-
clipart
Infograph
quickdraw
600
200
11
20
Figure 10: Number of samples per supercategory for each domain in DNAIL dataset.
21


Reviews:
[{"review_id": "u1RrQKxz1e", "reviewer": "Reviewer_n8Da", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "In general, I think the paper is quite interesting and should be valuable for the TMLR community.  However, before recommending acceptance, I would like to see some more baselines added just to understand the impact of the shape information on the final results.  If that issue is addressed, I would be happy to recommend acceptance.\n\n\nStrengths\n- The paper considers an important problem, from a perspective that I think has been fairly overlooked, i.e. cognitive sciences.\n- The results shown in the paper are quite strong. The proposed method outperforms the baselines by a significant margin, both on the previously established benchmark as well as on the new benchmark.\n- Additionally, the paper introduced the domain incremental learning dataset.  I think this is a valuable contribution that will be useful in further continual learning research.\n- The paper is pretty clearly written, easy to understand, and well explained.\n\n\nWeaknesses\n- I think the paper doesn't show why exactly the proposed method works well. In particular, it might be that the module that uses the shapes as the input makes the classification problem easier in general. As such, I would like to ask the authors to introduce two new baselines. The first one is a network that simply learns only from the shape data.  So basically the bias model trained in isolation on the continual learning task. Another baseline would be a network that uses both the RGB data and the shape data as input, without the multi-memory structure introduced in the paper. \n- Adding to the previous point, I think the author should try to understand how the shape information impacts the performance on any single task versus on the whole sequence. It might be that simply with the shape information we are able to better solve each task in separation and that's why we see the gains on the continual learning problem.\n- The novelty of this work is not very high.  In particular, it seems like an incremental improvement over the CLS-ER method proposed previously by Arani et al. who use a similar multi-network approach.  In this work, the authors also use the inductive bias network, which seems somewhat incremental. On the other hand, for this venue, high novelty is not an important criterion for accepting the paper, so I don't think this is a good enough reason to reject this paper. However, I think this point should be noted.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper considers the continual learning problem in ANNs from the perspective of cognitive sciences. According to cognitive sciences, humans are able to perform a wide variety of difficult tasks due to the structure of the cognitive system, in particular multi-memory systems in the brain and incorporation of cognitive biases. The idea of the paper is to use these concepts in artificial neural networks to build a system that performs well in continual learning. The authors propose a Dual Cognitive Architecture, DUCA, which consists of three neural networks: the working model, the semantic memory, and the implicit bias learner that focuses on the shape of the image rather than the full RGB picture. The authors show that the proposed model works well on the split CIFAR dataset and the split MNIST dataset as compared to strong baselines from continual learning literature. The authors propose their own dataset for domain incremental learning and they also show that the method performs quite well there.  Finally, the authors perform a wide variety of ablation studies showing a more thorough understanding of when and why the model works in practice.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please add the shape-based baselines outlined in the first two points of the \"Weaknesses\" section.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "vGo2MlcZyS", "reviewer": "Reviewer_jNRv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I think the main strength (or perhaps “selling point” is a better word) of this paper is the strong performance of the proposed method DUCA relative to established replay-based CL methods such as DER++ and Co2L. The suggestion that using shape information can have a specific benefit for continual learning is intriguing as well. Finally, I think that the proposed DN4IL dataset can be a useful contribution to the continual learning literature, albeit somewhat incremental.\n\nHowever, that the strong performance of DUCA is the main selling point, is in my opinion also an important weakness of this paper. The goal of continual learning research is not to achieve “state-of-the-art” performance on variants of Split CIFAR; rather the goal is to gain insights into how to build successful CL algorithms (and to eventually apply those to practical problems, although that goal is still distant). Controlled benchmarking experiments on variants of Split CIFAR could help towards this goal, but by themselves such experiments are of limited value, especially if there are concerns regarding how controlled they are.\n\nPerhaps the authors would argue that an insight from their paper is that inductive bias (or more specifically, a bias towards using shape information of images) can be beneficial for continual learning, but I’m afraid I do not think the current version of the paper satisfactorily demonstrates this. The authors show that using shape information can boost CL performance. However, as it is already known that using shape information can boost image classification (e.g., Gheiros et al, 2019 ICLR), it is not clear whether the use of shape information has benefits for CL on top of the general benefits it can have for image classification. I think the usage of shape information, and in particular the way this paper proposes to use shape information, *could* carry unique benefits for CL, but I’m afraid that the current version of this paper does not convincingly demonstrate this.\n(A related issue is that the use of shape information makes the benchmarking experiments less controlled. The reason that methods that are compared against do not use shape information might well not be because “they did not think to do so”, but rather because they wanted to test specific approaches in isolation.)\n\nAlthough the selected values for all hyperparameters are reported in Appendix D, it is not well described how these hyperparameters were selected for the different methods. It is stated that grid searches were performed, but it is not reported which hyperparameter values were considered for each method. Another drawback is that DUCA has relatively a lot of hyperparameters (which are set in a way that violates the CL principle of only seeing data in a certain sequence), more than most of the other methods that are compared with. Both of these are reasons contributing to why results of the benchmarking experiments by themselves are of limited value.\n\nThe paper presents DUCA as a “general” method for continual learning (among others by calling it a “framework for continual learning”). However, DUCA relies on the Sobel shape extraction algorithm, and as such its applicability seems to be limited to the image domain. Perhaps it could be argued that shape is merely used as an example of an inductive bias, and that for other modalities other inductive biases could be used. However, I do not think this is self-evident. I think either the authors should ensure it is clear that their proposed method is specific to the domain of images, or they should demonstrate that the underlying principle can also be successfully applied in other domains.\n\nOn page 6 the authors state: “As is evident from the different CL methods in the literature, the improvement in performance has been saturated on all variants of MNIST.” This is a very strong statement, but no references are provided for it. It also seems to me this statement is clearly wrong as I can think of many variations with MNIST for which performance has not saturated (e.g., when restricting memory, computation, amount of training samples). In fact, to strengthen this paper, I would encourage the authors to additionally include experiments on relatively simple datasets (for example MNIST), to provide results that are more easily reproducible and interpretable.\n\nThe way this paper describes class-, domain- and task-incremental learning seems rather simplistic and not in line with how these types of continual learning have been defined in Van de Ven et al. (2022, Nat Mach Intell). For example, on page 13 limitations of class-incremental learning are listed as “the assumption of the same number of classes across different tasks, the absence of reappearance of classes, and the sample distribution per class being well balanced”, but I don’t think these stated assumptions are part of the definition of class-incremental learning. Perhaps it could instead be argued that recent papers sometimes tend to make these assumptions when studying class-incremental learning?\nI think the authors should also be careful with calling these assumptions “limitations”. For example, for DN4IL, the authors make simplifications relative to DomainNet; it could be argued these simplifications make DN4IL “less realistic” than DomainNet, but the authors will probably agree that these simplifications are not “limitations”.\n\nMinor issues:\n- “class incremental learning” should be “class-incremental learning” (same with task- and domain-incremental learning, see for example https://www.grammarly.com/blog/hyphen/)\n- The first sentence of section 2.4 introduces “Cognitive Continual Learner (CCL)” but probably meant to say DUCA?\n- The cited Gheiros et al paper was published at ICLR 2019, not ICLR 2018", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Taking inspiration from cognitive science, this paper proposes a new multi-module architecture for continual learning. The proposed “Dual Cognitive Architecture” (or DULA) is made up of a working model (which is similar to a “standard” deep learning architecture), an inductive bias learner (which is trained on pre-processed shape information) and a semantic memory (which essentially is a delayed version of the working model). Several loss terms are introduced to encourage the outputs of these different modules to be comparable to each other. Using variants of the CIFAR10 and CIFAR100 datasets, the authors test the performance of the new architecture on task- and class-incremental learning settings; and for the domain-incremental learning setting a new dataset DN4IL is proposed, which is a curated subset of DomainNet. In all reported experiments, DUCA performs better than several established replay-based continual learning methods (such as DER++ and Co2L).", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I believe the main change that should be made is providing evidence that, and ideally also insights why, the various proposed aspects of DUCA have benefits that are specific for continual learning.\n\nOtherwise, please refer to the various raised issues and suggestions under “Strengths and Weaknesses”, also for more details on the requested change above.", "broader_impact_concerns": "No concerns in this regard.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "idmttxbLcQ", "reviewer": "Reviewer_3ogS", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The method seems novel.\n\n- It seems to work well.\n\n- The new dataset is potentially interesting.\n\nWeaknesses:\n\nThere doesn't seem to be any deadly flaw in the proposed method, but some question marks.\n\n- From the results,  it seems that the addition of the shape-based model (trained on edge-detection images) was the main factor in the approach's superior performance (see especially  section 6.5 on ablations). Thus, the main  result of the paper may simply be that image-based tasks benefit from a bias towards shapes. This may be interesting, but it may also seem somewhat orthogonal to  the actual problem of lifelong learning / CL. \n\n- While the authors address the complexity of the various methods, they do not address the number of trainable parameters (unless I missed it). Are the comparison between models with equal numbers of trainable parameters and wallclock time?\n\n- The system includes a dedicated  network trained on edge-detected images. This plays a large part in the model's performance. But is this improvement caused by the architectural innovation, or simply the inclusion of edge-detected /shape-emphasized images  in the data? An alternative would be to augment main training set with edge-detected images, using only the main network and the \"semantic\" (slow-update)  network in the  model, without including a  dedicated \"inductive bias\" network. How would this simpler method compare with the proposed approach, all  other things (training time, number of parameters, etc.) being equal?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The author propose a novel method for continual learning  (CL), which is both rehearsal- and  architecture-based. \n\nThe method consists in augmenting the main network with a\"semantic memory\" (a running average of the main network updated at random times) and an \"inductive bias\" system (a network  trained on edge-detected versions of the input images, presumably emphasizing shape information). The subsystems are trained on both current examples and previous examples, stored in a replay buffer. IIIUC the outputs of the various networks are constrained to be similar by MSE losses (Eqs. 3,4). The method is tested on various continual learning  problems, representing Task-, Domain- and Class-incremental  learning, and shows significant benefits over existing approaches. \n\nIn addition, a new dataset for domain-incremental learning (essentially curated form a larger existing dataset) is introduced.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Please add number of trainable parameters and if possible wallclock training time for the various approaches.\n\n- At the very least, explain the benefit of adding  a dedicated shape-based  network to  the system, rather than merely augmenting the main dataset with edge-detected images. Actual experiments would be best, if  at all possible.\n\n- Section 2.4, second paragraph: what  are the \"encoder\" and the \"classifier\"? This does not seem to have been discussed beforehand.\n\n- Last paragraph of p. 1: fix the second sentence.\n\n- Last paragraph of p.3: \"deferentially\" -> \"differentially\".", "broader_impact_concerns": "I do not see any broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: On balance, I believe that the authors have provided enough evidence to support their core claims and addressed most of the reviewer comments. However, I believe that there are still two important minor modifications that need to be made:

1) The abstract is not clear enough on what the model actually includes. Specifically, nowhere in the abstract is it noted that what the authors mean by "multiple memory systems" is a slow and fast learner for rehearsal (in-line with complementary learning systems theory), and that what they mean by "inductive biases" is specifically a shape bias. As such, a reader will not really understand the core model design from the abstract, even at a high level, so it is too vague. The authors should fix this before publication.

2) Though the authors inserted some language about moving this beyond the image domain, it is, in my judgement, insufficient. Specifically, the authors say, "...the underlying principles and mechanisms of DUCA can be adapted to different modalities. The DUCA framework can be extended to other domains by selecting appropriate inductive biases relevant to those domains." But, what would "appropriate inductive biases" be in other domains? Also, why would the other components of DUCA help in other domains? A few other sentences to suggest potential future directions to extend this work beyond the image domain would greatly help the conclusion of the paper.
Decision: Accept (poster)