A Cognitive-inspired Multi-Module Architecture for Continual Learning
Authors: Shruthi Gowda
Bahram Zonooz
Elahe Arani
Venue: Submitted to ICLR 2023 (2023)
Keywords: Continual Learning
Catastrophic Forgetting
Experience Replay
Lifelong Learning
Cognitive Learning
Inductive Bias

Abstract: Artificial neural networks (ANNs) exhibit a narrow scope of expertise on stationary independent data. However, data in the real world is continuous and dynamic, and ANNs must adapt to novel scenarios while also retaining the learned knowledge to become lifelong learners. The ability of humans to excel at these tasks can be attributed to multiple factors ranging from cognitive computational structures, cognitive biases, and the multi-memory systems in the brain. We incorporate key concepts from each of these to design a cognitive-inspired continual learning method. Cognitive Continual Learner (CCL) includes multiple modules, implicit and explicit knowledge representation dichotomy, inductive bias, and a multi-memory system. CCL shows improvement across all continual learning settings and also exhibits reduced task recency bias. To test versatility of continual learning methods on a challenging distribution shift, we introduce a novel domain-incremental dataset Domain${^2}$IL. In addition to improved performance on existing benchmarks, CCL also demonstrates superior performance on this dataset.

Under review as a conference paper at ICLR 2023
A COGNITIVE-INSPIED MULTI-MODULE ARCHITEC- TURE FOR CONTINUAL LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
Artificial neural networks (ANNs) exhibit narrow scope of expertise station- ary independent data. However, data in the real world is continuous and dynamic, and ANNs must adapt to novel scenarios while also retaining the learned knowl- edge to become lifelong learners. The ability of humans to excel at these tasks can be attributed to multiple factors ranging from cognitive computational struc- tures, cognitive biases, and the multi-memory systems in the brain. We incor- porate key concepts from each of these to design a cognitive-inspired continual learning method. Cognitive Continual Learner (CCL.) includes multiple modules, implicit and explicit knowledge representation dichotomy, inductive bias, and a mulli- system. CCL shows improvement across different settings and also shows a reduced task recency bias. To test versatility of continual learning meth- ods on challenging distribution shift, we introduce novel dataset DN4IL. In addition to improved performance on existing benchmarks, CCL also demonstrates superior performance on this dataset
INTRODUCTION
Deep learning has seen rapid progress in recent years, and supervised learning agents have achieved superior performance in perception tasks. However, unlike a supervised setting, where data is static, and independent and identically distributed, real-world data is changing dynamically. Continual learning (CL) aims learning multiple tasks when data is streamed sequentially (Parisi al. 2019). This is crucial in real-world deployment settings, as the model needs to adapt quickly to novel data (plasticity), while also retaining previously learned knowledge (stability). Artificial neural networks (ANN), however, are still not effective continual learners as they often fail to generalize to small changes in distribution and also suffer from forgetting old information when presented with new data (catastrophic forgetting) (McCClskey & Cohen| 1989).
Humans, on the other hand, show a better ability to acquire new skills while also retaining previously learned skills to greater extent. This intelligence can be attributed to different factors in human cog- nition. Multiple theories have been proposed to formulate an overall cognitive architecture, which is broad Domain-generic cognitive computation model that captures the essential structure and pro- cess of the mind. Some of these theories hypothesize that, instead of a single standalone module, multiple modules in the brain share information to excel at particular task. CLARION (Connec- tionist learning with rule induction online) (Sun & Franklin (007) is one such theory that postulates an integrative cognitive architecture, consisting of a number of distinct subsystems. It predicates a dual representational structure (Chaiken & Trope, 1999), where the top level encodes conscious explicit knowledge, while the other encodes indirect implicit information. The two systems interact, share knowledge, and cooperate in solving tasks. Delving into these underlying architectures and formulating a new design can help in the quest of building intelligent agents.
Multiple modules can be instituted instead of single feedforward network. An explicit module that learns from the standard visual input and an implicit module that shares indirect contextual knowledge. The implicit module can be further divided into more sub-modules, each providing dif- ferent information. Inductive biases and semantic memories can act as different kinds of implicit knowledge. Inductive biases pre-stored templates or knowledge that provide some meaning- ful disposition toward adapting to the continuously evolving world (Chollet 2019). Furthermore,
Code and the DN4II. dataset will be made accessible upon acceptance.

---

Under review as a conference paper at ICLR 2023
theories (Kumaran et al. 2016) postulate that after rapidly learning information, gradual consol- idation of knowledge transpires in the brain for slow learning of structured information. Thus, the new design incorporates multiple concepts of cognition architectures, the dichotomy of implicit and explicit representations, inductive biases, and multi-memory systems theory.
To this end, we propose Cognitive Continual Learner (CCL), malti-module architecture for CL. The explicit working module processes the standard input data. Two different sub-modules are in- troduced for the implicit module. The inductive bias learner embeds relevant prior information, and as networks are shown to be biased toward textural information (unlike humans that are more biased toward global semanniss)(Geirros et al., 2018), we propose to utilize the global shape information as the prior. Shape is already present in the visual data but in an indirect way, and extracting this im- plicit information and sharing with the explicit module will help to learn more generic and high-level representations. Further, to emulate the consolidation of information in the slow-fast multi-memory system, a gradual accumulation of knowledge from the explicit working module is embedded in the second semantic memory sub-module. We show that interacting and leveraging information be- tween these modules can help alleviate catastrophic forgetting while also increasing the robustness to distribution shift.
CCL achieves superior performance across all CL settings on various datasets. CCL outperforms the SOTA CL methods on Seq-CIIFARO,, 10, Seq-CIFARION in the class incremental settings. Furthermore, in more realistic general class incremental settings where the task boundary is blurry and classes are not disjoint, CCL shows significant gains. The addition of inductive bias and semantic memory helps to achieve a better balance between the plasticity-stabilis trade-off. The prior in the form of shape helps produce generic representations, and this results in CCL exhibiting a reduced task-recency bias. Furthermore, CCL also shows higher robustness against natural corruptions. Finally, to test the capability of the CL methods against distribution shift, we introduce a domain incremental learning dataset, DN4IL, which is carefully designed subset of the DomainNet dataset (Peng et al. 2019). CCL shows considerable robustness across all domains on these challenging data, thus establishing the efficacy of our cognitive-inspired CI. architecture. Our contributions are as follows:
Cognitive Continual Learner (CCL), novel method that incorporates aspects of cognitive architectures, multi-memory systems, and inductive bias into the CL framework.
Introducing DN4IL, a challenging domain incremental learning dataset for CL.
Benchmarks across different CL settings: class incremental, task incremental, generalized class incremental, and domain incremental learning.
Analyses the plasticity-stabiliy trade-off, task recency bias, and robustness to natural corruptions.
2 METHODOLOGY
2.1 COGNITIVE ARCHITECTURES
Cognitive architectures refer computational models that encapsulate the overall structure of the cognitive process in the brain. The underlying infrastructure of such a model can be leveraged to develop better intelligent systems. Global workspace theory (GWT) (Juliani al., (022) postulates that human cognition is composed of multitude of special-purpose processors and is not a single standalone module. Different sub-modules might encode different contextual information which, when activated, can transfer knowledge to the conscious central workspace to influence and help make better decisions. Furthermore, CLARION (Sun & Franklin 2007) posits dual- duual-ysstem cog- nitive architecture with two levels of knowledge representation. The explicit module encodes direct knowledge that is externally accessible. The implicit module encodes indirect knowledge that not directly accessible, but can be obtained through some intermediate interpretive or transformational steps. These two modules interact with each other by transferring knowledge between each other.
Inspired by these theories, we formulate method that incorporates some of the key aspects of cognitive architecture into the CL method. A working module, which encodes the direct sensory data, forms the explicit module. A second module that encodes indirect and interpretive information forms the implicit module. The implicit module further includes multiple sub-modules to encode different types of knowledge.
2

---

Moreover, many theories have postulated that intelligent agent must possess deferentially spe- cialized learning memory systems (Kumaran et al. 2016). While one system rapidly learns the individual experience, the other gradually assimilates the knowledge. To emulate this behavior, Under review as a conference paper at ICLR 2023
Explicit Module
Implicit Module
Cognitive: Explicit Learning
Knowledge Sharing
Cognitive: Implicit Learning
Knowledge: Direct
Knowledge: Indirect
Source of knowledge: External, Precise
Source knowledge:
Assimilated, Contextual
Episadic Memory
Inductive Blas Learner
Sensory Data
Cuxs
Wor king Model
LKS
Semantic Memory
LxSwa
Stochastic
Momentunn
Update
Figure 1: Schematic of Cognitive Continual Learner (CCL). The working model in explicit module learns direct sensory data. Within the implicit module, the inductive bias learner encodes the prior shape knowledge and the semantic memory consolidates information from the explicit module.
2.2 INDUCTIVE BIAS
The sub-modules in the implicit module need to encapsulate implicit information that can provide more contextual and high-level supervision. One such knowledge can be prior knowledge or inductive bias. Inductive biases prestored templates that exist implicitly even in earlier stages of the human brain (Pearl Mackenzic 2018). For instance, cognitive inductive bias may one of the reasons why humans focus the global semantics of objects to make predictions. ANNs, on the other hand, are more prone to rely on local cues and textures (Geirhos et al.) 2018). Global semantics shape information already exists in the visual data, but utilize shape as indirect information in the implicit module. The sub-module uses a transformation step to extract the shape and share this inductive bias with the working module. As the standard (RGB) image and its shape counterpart can be viewed as different perspeciives/modaliies of the same data, ensuring that the representation of one modality is consistent with the other increases robustness to spurious correlations that might exist in only one of them.
indirect way. Hence,
2.3 MULTI MEMORY SYSTEM
establish second sub-module that slowly consolidates the knowledge from the working module.
2.4 FORMULATION
To this end, propose novel method Cognitive Continual Learner (CCL), which incorporates all these concepts into the CL paradigm. CCL consists of two modules, the explicit module and the implicit module. The explicit module has single working model and processes the incoming
3

---

Under review as a conference paper at ICLR 2023
direct visual data. The implicit module further consists of two sub-modules, namely the inductive bias learnr and the semantic memory. They share relevant contextual information and assimilated knowledge with the explicit module, respectively. Figure Sigure[shows shows the overall architecture.
In the implicit module, semantic memory NsM. consolidates knowledge at stochastic intervals from the working model Nw in the explicit module. The other sub-module, the inductive bias learner NIBL, processes the data and extracts the shape information (Section [B) NWM processes the RGB data, Nsw consolidates the information from the working module an update frequency in stochastic manner, and BI. learns from the shape data. f represents the combination of the encoder and the classifier, and 8wMs M, ASM. and BI are the parameters of the three networks.
A CL classification consists of sequence of T tasks and, during each task t E 1, 2..., samples Te and their corresponding labels ze are drawn from the current task data Dr- Furthermore, for each subsequent task, a random batch of exemplars is sampled from episodic memory B as 24. An inductive bias (shape) filter applied to generate shape samples, Ic, = [B(,)) and æbb, (x²)) Reservoir sampling (Vitter 1985) is incorporated to replay previous samples. Each of the networks NwM and NI learns its own modality with supervised cross-entropy loss on both the current samples and the buffer samples:
ve) Lor(f(rs fw (wwas..))) Lsupwm = LCE(f(xe) (wllx:00m))) aw)
SSupIa = ( uc) + Lc
(2)
The Knowledge Sharing (KS) objectives are designed to transfer and share information between all modules. KS occurs for current samples and buffered samples. We employ the mean squared error as the objective function for all KS losses. To provide shape supervision to the working model and vice versa, bidirectional decision space similarity constraint (Cuis)) enforced to align the features of the two modules.
LaKS
E a~D
(1ss0)0L)) - (2; 01
The consolidated structural information in semantic memory is transferred to both the working model and the inductive bias learner by aligning the output space on the buffer samples, which further helps in information retention. The loss functions LK and LKSIBL are as follows;
LKSWM E (1 Osas) - [116; M
(4)
zw~B
LKSIBL = TE (1); OSM) - ( =
(5)
Thus, the overall loss functions for the working model and the inductive bias learner are as follows;
LWM Lsupwar M + ALous + XLLsSw LIBI. Lsuprat + +YLiis 7 LKSsaL
(6)
The semantic memory of the implicit module is updated with a stochastic momentum update (SMU)
of the weights of the working model at rate with decay factor of d,
OsM = d OsM + (1 - d) OwM if (O0,1) <r
(8)
More details are provided in Algorithm 2
Note that we use semantic memory (0sM) for inference,
as it contains consolidated knowledge across all tasks.
3 EXPERIMENTAL SETTINGS
ResNet-18 (He et al. 2016) architecture used for all experiments. All networks are trained us- ing the SGD optimizer with standard augmentations of random crop and random horizontal flip. The different hyperparameters, tuned per dataset, are provided in D The different CL settings are explained in detail in Section We consider CLass-IL, Domain-IL and also report the Task-IL -LL set- tings. Seq-CIFAR10 10 and Seq-CIFAR100 (Krizhevsky et al.) (209) for the class incremental learning (Class-IL) settings, which are divided into 5 tasks each. As addition Class-IL, also sider and evaluate General Class-IL- (GCIL) (Mi et al. (020) on CIFAR100 dataset. For the domain incremental learning (Domain-IL.), we propose novel dataset, DN4II..
4

---

Under review as a conference paper at ICLR 2023
Table Comparison of different methods on standard CL benchmarks (Class-IL, Task-IL and GCIL settings). CCL shows consistent improvement over all methods for both buffer sizes.
Seq-CIFAR10
Seq- CIF-CIIFRROO
GCIL-CIFAR 100
Method
Class-IL
Task-IL
Class-IL
Task-IL
Uniform
Longtail
JOINT
92.20 19.62
98.31
70.62:1064
86. 16.49
60.45
60.10
SGD
61.02.13.3
17.58.10.04
40.4610.99
10.36.10.13
9.62.022
ER
44,79
91.19+0.94
21.40++0.22
61.36
16.522t0.00
16. 16.30=0.30
DER+
1.18
91.92=00.0
29.60=11144
62.49
27.73=0093
26.48=2204
Co'L
1..57
93.43
31.90110.38
55.02
200
ER-ACE
62.08
92.20
32.49:10.95
59.77
27.64
25.10#2264
CLS-ER
0..75
93.90#00.6
1..9
73.49
35.88:1041
35.67=0.72
CCL
70.04=1.07
94.49
46.55
76.66=0.41
39.61:10.0
38.94=0.16
ER
57.74
93.61 +0.27
28.02.10.31
68.23 16.
23.62
22.36-11.27
DER+
1.370
93.88
41.40.10.6
70.61 10.11
35.80
34.23
Co'L
74.7
95.90.10.26
39.21
62.98 10.58
500
ER-ACE
68.45
93.47.1100
40.67 1:006
66.45 10.71
30.14.1.11
31.88 10.73
CLS-ER
75.22=0071
94.94
51.40=1.00
78. 18.
38.94
38.79
CCL
76.20
95.95:t0.14
53.23:+162
80.12
44.25
42.75-10.18
RESULTS
We provide a comparison of our method with standard baselines and multiple other SOTA CL meth- ods. The lower and upper bounds reported SGD (standard training) and JOINT (training all tasks together), respectively. We compare with other rehearsal- based methods in the literature, namely ER, DER (Buzzega et 2020). Co (Cha al. 2021). ER-ACE (Caccia al. 2021 and CLS-ER (Arani et al. 2021). Table S2 shows the average performance in different settings over three seeds. Co-L utilizes task boundary information, and therefore the GCIL setting is not applica- ble. The results are taken from the original papers and, if not available, using the original codes, we conducted hyperparameter search for the new settings.
CCL achieves the best performance across all datasets in all settings. In the challenging Class-IL setting, we observe a gain of ~50 over DER++, thus showing the efficacy of adding multiple modules to CL. Furthermore, we report improvements of ~6% on both the Seq-CIFAR10 and Seq- CIFARI00 datasets, over CLS-ER, which utilizes two memories in its design. CCL has a single semantic memory, and the additional boost is procured by prior knowledge from the inductive bias leamner Improvement is prominent even when the memory budget is low (200 buffer size). GCIL represents
realistic setting, the task boundaries overlap in any task. GCIL-Longtai version also introduces an imbalance in the sample distribu- tion. CCL shows significant improvement tion from the inductive bias learner offers the global high-level context, which helps in producing generic representations that are not biased towards learning only the current task hand. Further- more, sharing of the knowledge that has been assimilated through the appearance of overlapping classes through the training scheme, further facilities learning in this general setting. The overall results indicate that the dual knowledge sharing between the explicit working module and the im- plicit inductive bias and semantic memory modules enables both better adaptation new tasks and information retention.
blurry and classes reappear and
both versions of
Shape informa-
5 DOMAIN INCREMENTAL LEARNING
Intelligent agents deployed in real-world applications need to maintain consistent performance through changes in the data and enviromment. Domain-IL aims to assess the robustness of the CL methods the distribution shift. In Domain-IL, the classes in each task remain the same, but the input distribution changes, and this makes for a more plausible use case for evaluation. However, the datasets used in the literature do not fully reflect this setting. For instance, the most common
5

---

Under review as a conference paper at ICLR 2023
ER DER++
CLS-ER
70
Plasticity
Stability
50
CCL
60
40
50
Accuracy
Accuracy
40
30
30
20
10
[B|-200
[B|-500
ER
DER+
CLS-ER
CCL
Figure 2: Accuracy (left) and plasticity-staid stability analysis (right) on DN4IL dataset. CCL substan- tially outperforms other methods and demonstrates better plasticity-stabiliy trade-off.
datasets used in the literature are different variations (Rotated and Permuted) of the MNIST dataset (LeCun et al. 1998). MNIST is a simple dataset, usually evaluated on MLP networks, and its variations do reflect the real-world distribution shift challenges that CL method faces. As is evident from the different CL methods in the literature, the improvement in performance has been saturated all variants MNIST. Farquhar & Gal (2018) propose fundamental desiderata for CL evaluations and datasets based on real-world use cases. One of the criteria is to possess cross-task resemblances, which Permuted-NNIST clearly violates. Thus, a different dataset needed test the overall capability of CL method to handle the distributional shift.
5.1 DN4IL DATASET
To this end, we propose DN4IL (DomainNet for Domain-IL), which is a well-crafted subset of the standard DomainNet dataset (Peng al., 2019), used in domain adaptation. DomainNet consists of common objects in six different domains real, clipart, infograph, painting, quickdraw, and sketch. The original DomainNet consists of 59k samples with 345 classes in each domain. The classes have redundancy, and moreover, evaluating the whole dataset can be computationally expensive in CL setting. Considering different criteria such the relevance of classes, uniform sample distribution, computational complexity, and ease of benchmarking for CL. we create the version DN4IL, which tailor-man made for continual learning.
All classes were grouped into semantically similar supercategories. Out of these, subset of classes was selected that had relevance to domain shift while also having maximum overlap with other standard datasets such as CIFAR-10 and CIFAR-100, as this can facilitate in performing out-of- distribution analyses. 20 supercategories were chosen with classes each (resulting in a total of 100 classes). In addition, to provide balanced dataset, performed class-wise sampling. First, we sample images per class each supercategory and maintain class balance. Second, we choose samples per domain, that it results in dataset that has near-uniform distribution across all classes and domains. The final dataset DN4IL succinct, more balanced, and more computationally efficient for benchmarking, thus facilitating research in CL. Additionally, the new dataset deems more plausible for real-world settings and also adheres to all the evaluation desiderata by (Farquhar) & Gal 2018). The challenging distribution shift between domains provides an apt dataset to the capability of CL methods in the Domain-ll IL setting. More details, statistics, and visual examples of this crafted dataset are provided Section
5.2 DN4IL PERFORMANCE
Figure 2 (left) reports the results on DN4IL for two different buffer sizes (Values provided in Table S8 CCL shows a considerable performance gain in the average accuracy across all domains and can be primarily attributed to the supervision from the shape data. Standard networks tend to exhibit texture bias and learn background or spurious cues (Geirhos et al.] 2018) that result in per- formance degradation when the distribution changes. Learning global shape information of objects,

---

Under review as a conference paper at ICLR 2023
ER
DER+++
CLS-ER
CCL
After real 59.3
62.3
52.7
60.5
After clip 64.1
54.2
57.2
51.5
61.4
After graph
27.0
54.0
59.0
62.1
26,
After paint 15.0
51.2
19.1
42.6
60.7
52.8
After sketch 40.4
43.1
11.8
29.9
58.4
50.0
53.6
16.8
39.9
59.5
55.2
54.7
21.0
47.6
49.8
59.7
63.
23.7
61.8
61.3
After qdraw 27.6
30.8 8.0
20.2
28.6
71.1
435
16.9
13.6
30.6
68.9
51.1
51.9
10.8
38.1
60.9
52.1
54.9
61.4
19
64.5
56.6
61.0
real
dlip
graph
paint
sketch
gdraw
real
clip
graph paint
sketch
qdraw
real
clip
graph
paint
sketch
qdraw
real
cli
graph
paint sketch
gdraw
Figure 3: Tise performance on DN4IL (B))500), where each task represents a domain. CCL shows more retention of old information without compromising much on current accuracy.
on the other hand, helps in learning generic features that can translate well to other distributions. Semantic memory further helps consolidate information across domains. Maintaining consistent performance to such difficult distribution shift proves beneficial in real-world applications, and the proficiency of CCL this setting can thus open up new avenues for research cognition-inspired multi-module architectures.
6 ANALYSIS
6.1 CESTICITY-STABILIIA TRADE-OFF
Plasticity refers to the capability of a model to learn new tasks, while stability shows how well can retain old information. The plasticity-stabiie
dilemma is long-standing problem in CL, which requires an optimal balance between the two. We measure each of these to assess the competence of the CL methods. Plasticity is computed as the average performance of cach task when it is first learned (e.g.. the accuracy of the network trained on task T2, evaluated on the test set of T2). Stability is computed as the average performance of all tasks 1:T-1, after learning the final task T. Figure 2 (right) reports these numbers for the DN4IL dataset. As seen, the ER and DER methods exhibit forgetting and show low stability and concentrate only the newer tasks. CLS-ER shows greater stability, but at the cost of reduced plasticity. However, CCL shows the highest stability while maintaining comparable plasticity. The shape knowledge in CCL helps in learning generic solutions that can translate to new tasks, while the semantic consolidation update at stochastic rates acts as a regularization to maintain stable parameter updates. Thus, CCL. strikes better balance between plasticity and stability.
6.2 TASK-WISE PERFORMANCE
The average accuracy across all tasks does not provide a complete measure of the ability of a net- work to retain old information while learning new tasks. To better represent the plasticity-stability measure, we report the task-wise performance at the end of each task. After training each task, measure the accuracy on the tasks of DN4IL. The last row represents the performance of each task after the training is complete. ER and DER++ show performance degradation on earlier tasks, as the model continues training on newer tasks. Both perform well on the last task and display the lowest stability. CCL reports the highest information retention on older tasks, while also maintaining plasticity. For example, the accuracy on the first task (real) reduces to 27.6 on ER after training the 6 tasks (domains), while the CCL maintains the accuracy of 54.9. CL.S-ER shows better retention of old information but the cost of plasticity. The last task CLS-ER shows lower performance compared to CCL (52.1 61.0). Similar trend (with more gains) is seen on Seq- Eeq-CIFARIO dataset Appendix Figure S2 The performance of the current task in CCL is relatively lesser and can be attributed to the stochastic update rate of this model.
of each of the previous tasks. Figure reports this for all
To shed more light the performance of each of the modules in CCL, also provide the perfor- mance of the working model and the inductive bias learner, in Appendix Figure ST]] The working model shows better plasticity, while CCL. (semantic memory) displays better stability. Overall, all

---

Under review as a conference paper at ICLR 2023
Task
DER
CLS-ER CLL
0.5
Task
Robustness (Corruption)
22.5
Task Probablity
0.4
17.5
0.3
12.5
10.0
0.1
0.0
ER
DER++ CLS-ER
CCL
5.0
Method
Severity
Figure 4: CCL shows reduced task recency bias (left), as well as higher robustness against natural corruption (right) Beq-CIFARIO (|3)=200) dataset.
the modules in the proposed approach present unique attributes that improve the learning process and improve performance and reduce catastrophic forgetting.
6.3 RECENCY-BIAS ANALYSIS
Recency bias is behavior in which the model predictions tend to be biased toward the current the most recent task (Wu al. 2019). This is undesirable CL model, as it results in biased solution evaluate the models on the test set (of all tasks) and calculate the probability of predicting each task. The output distribution for each test sample is computed for all classes, and the probabilities are averaged per task.
that forgets the old tasks. To this end, after the end of the training,
Figure (left) shows the probabilities for each task on Seq- CIF-CIF dataset. As shown, the ER and DER++ methods tend to incline most of their predictions towards the classes task, thus creating misguided bias. CCL shows a lesser bias compared to both of these baselines. CLS-ER exhibits reduced bias due to the presence of multiple memories. but the distribution is still relatively skewed (w..t. probability of 0.2). CCL shows more of a uniform distribution across all tasks. The dual information from the shape data and the consolidated knowledge in breaking away from the Occam's 's razor pattern of neural networks to default to the easiest solution.
in the last
tasks helps
6.4 ROBUSTNESS
Lifelong agents, when deployed in real-world settings, must be resistant to various factors, such as lighting conditions, changes weather, and other effects of digital imaging. Inconsistency in predictions under different conditions might result in undesirable outcomes, especially in safety- critical applications such as autonomous driving.
measure the robustness of the CL method against such natural corruptions, created dataset by applying fifteen different corruptions (Table (S), at varying levels of severity (1- least severe to most severe corruption).
The performances the fifteen corruptions averaged at each severity level and shown in Figure 4 (right). CCL. outperforms all other techniques at all severity levels. ER, DER++ and CLSS-RR show fast decline in accuracy severity increases, while CCL maintains stable perfor- mance throughout. Implicit shape information provides different perspective of the same data to the model, which helps generate high-level, robust representations. CCL, along with improved continual learning performance, also exhibits improved robustness to corruptions, thus proving to be a better candidate for deployment real-world applications.
6.5 ABLATION STUDY
CCL architecture comprises multiple components, each contributing to the efficacy of the method. The explicit module has the working model, and the implicit module has semantic memory (SM) and inductive bias learner (IBL). Disentangling different components in the CCL, can provide more insight into the contribution of each of them to the overall performance.
8

---

Under review as a conference paper at ICLR 2023
Table 2: Ablation to analyse the effect of each component of CCL on Seq-CIFARIO and DN4IL.
SM
IBL.
KS (WM++IBL)
Req-CIFARIO
DN4IL
70.04
44.23
1
X
69.28 19.28.1133
40.35.10.4
69.2111146
39.76.00.6
X
/
V
64.61
37.33
X
44.79+1.86
26,59+0.31
Table reports the ablation study w.r.t to each of these components on both Seq-CIFARIO and DN4IL datasets. Considering the more complex DN4IL dataset, the ER accuracy without any of our components is 26.59, Adding cognitive bias (IBL) improves performance by 40%. Shape informa- tion plays prominent role, as the networks need to learn the global semantics of the objects, rather than background or spurious textural information to translate performance across domains. Adding the dual-memory component (SM) shows an increase of approximately 49% over the vanilla base- line. Furthermore, KS between explicit and implicit modules current experiences also plays key role in performance gain. Combining both of these cognitive components and, in general, following the multi-module design shows gain of 66%. A similar trend is seen Beq-CIFARIO.
7 RELATED WORKS
Athearsal-based approaches. which revisit examples from the past to alleviate catastrophic forget- ting. have been effective in challenging CL scenarios (Farquhar & Gal 2018). Experience Replay (ER) (Riemer al. 2018) methods use episodic memory to retain previously seen samples for re- play purposes. DER ++ (Buzzega et al.] 2020) adds consistency loss on logits, in addition to the ER strategy. CO (Cha et al. 2021 uses contrastive learning from the self-supervised learning domain to generate transferable representations. ER-ACE (Caccia al. 2021) targets the representation drift problem in online CL and develops a technique to use separate losses for current and buffer samples. All of these methods limit the architecture to single stand-alone network, contrary the biological workings the brain. CLS-ER (Arani et al., (021) proposed a multi-network approach that emulates fast and slow learning systems by using semantic memories, each aggregating weights at different times. Though CLS-ER utilizes the multi-memory design, sharing of different kinds of knowledge is not leveraged, and hence presents method with limited scope. CCL. di- gresses from the standard architectures and proposed a multi-module design that is inspired by the cognitive computational architectures. It incorporates multiple sub-modules, each sharing different knowledge to develop an effective continual learner that has better generalization and robustness.
8 CONCLUSION
We introduced novel framework for continual learning which incorporates concepts inspired by cognitive architectures, high-level cognitive biases, and the multi-memory system. Our method, Cognitive Continual Learner (CCL), includes multiple subsystems with dual knowledge representa- tion. CCL designed a dichotomy of explicit and implicit modules in which information is selected, maintained, and shared with each other, to enable better generalization and robustness. CCL out- performed on Seq-CIFAR 10 and Seq-CIFAR 100 on the Class-IL, setting. In addition, it also showed significant gain in the more realistic and challenging GCIL setting, Through different analyses, we showed the better plastictty-taabiliy balance achieved by CCL. Furthermore, shape prior and knowl- edge consolidation helps to learn more generic solutions, indicated by the reduced task recency bias problem and higher robustness against natural corruptions. Furthermore, we introduced a challeng- ing domain-IL. dataset, DN4IL, with six disparate domains. The significant improvement of CCL on this complex distribution shift demonstrates the benefits of shape context, which helps the network to converge on a generic solution, rather than a simple texture- biased one. In general, incorporating design inspired by the cognitive model and sharing information between explicit and implicit in- ductive bias and implicit semantic memory modules, instead of standalone network, helps enhance lifelong learning, while also improving generalization and robustness.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general contin- ual learning method based on complementary learning system. In International Conference on Learning Representations, 2021.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark ex- perience for general continual learning: strong, simple baseline. Advances in neural information processing systems, 101115592019995 2020.
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. In Interna- tional Conference on Learning Representations, 2021.
Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co21:
Contrastive continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9516-9525, 2021.
Shelly Chaiken and Yaacov Trope. Dual-process theories in social psychology. Guilford Press, 1999.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Picter Abbeel. Info- gan: Interpretable representation learning by information maximizing generative adversarial nets. Advances in neural information processing systems, 29, 2016.
François Chollet. On the measure of intelligence. arXiv preprint arXiv: 1911.0157,1 2019.
Lijun Ding and Ardeshir Goshtasby. On the canny edge detector. Pattern Recognition, 34(3):721- 725, 2001.
Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning. 2018.
Robert Geirhos, Patricia Rubisch, Claudio Michaelis. Matthias Bethge, Felix A Wichmann, and Wieland Brendel. Imagenet-trained cnns are biased towards texture; increasing shape bias im- proves accuracy and robustness. In International Conference on Learning Representations, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Arthur Juliani, Kai Arulkumaran, Shuntaro Sasai, and Ryota Kanai. On the link between conscious function and general intelligence humans and machines. arXiv preprint arXiv:2204.05133, 2022.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Dharshan Kumaran, Demis Hassabis, and James McClelland. What learning systems do intelligent agents need? complementary learning systems theory updated. Trends in cognitive sciences, 20 (7):512-534, 2016.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 (6(11):2278-233 1998.
Yingwei Li, Qihang Yu, Mingxing Tan, Jieru Mei, Peng Tang, Wei Shen, Alan Yuille, et al. Shape- texture debiased neural network training. In International Conference on Learning Representa- tions, 2020.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
10

---

Under review as a conference paper at ICLR 2023
Fei Mi, Lingjing Kong, Tao Lin, Kaicheng Yu, and Boi Faltings. Generalized class incremental learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- fion Workshops, pp. 240 2400-2411220 2020.
German Parisi, Ronald Kemker, Jose Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-71, 2019.
Judea Pearl and Dana Mackenzie. The book of why: the new science of cause and effect. Basic books, 2018
Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang. Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 1406-1415, 2019.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning learn without forgetting by maximizing transfer and minimizing interfer- ence. In International Conference on Learning Representations, 2018.
Irwin Sobel and Gary Feldman. A 3x3 isotropic gradient operator for image processing. talk at the Stanford Artificial Project in, pp. 271-272, 1968.
Ron Sun and Stan Franklin. Computational models consciousness: A taxonomy and some exam- ples., 2007.
Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. arXiv preprint arXiv:1904.07734, 2019.
Jeffrey S Vitter. Random sampling with a reservoir. ACM Transactions on Mathematical Software (TOMS), 11(1)337-57, 1985.
Yue Wu, Yinpeng Chen, Lijuan Wang. Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 374-382, 2019.

---

Under review as a conference paper at ICLR 2023
A
APPENDIX
A.1 CCL
Figure ST] presents the task-wise performance of all the three networks in the CCL architecture, on DN4IL dataset. Semantic memory helps in information retention by maintaining high accuracy older tasks and is more stable. The performance of the current task relatively lower than that of the working model and could be due to the stochastic update rate of this model. The working model has better performance on new tasks and is more plastic. Inductive bias leaner is evaluated on the transformed data (shape) and also achieves balance between plasticity and stability. In general, all modules our proposed method present unique attributes that improve the learning process by improving performance and reducing catastrophic forgetting.
Semantic Memory (Default)
Working Model
Inductive Bias Learner
After
60.5
54.5
After T2
62.9 61.4
57.0
71.2
46.5
65.4
After
73.09.0
62.1
26.9
50.88
-39.7
22.3
After T4
60.1
60.7
26.6
52.8
55.2
54.8
21.5
55.5
42.0
44.8
15.4
40.3
After T6
54.9
61.4
19.0
44.5
56.6
61.0
47.3
54.2
14.3
36.6
47.7
71.4
38.2
48.4
11.5
29.0
46.2
71.2
T2
T4
T5
T1
T2
T3
T5
T3
Figure S1: Task probability analysis of all CCL components DN4IL dataset with 500 buffer size. Semantic memory displays better stability while working model displays better plasticity.
Figure S2 presents a similar analysis on the Seq-CIFAR10 dataset. The trend is similar, but the performance gain is much higher on this dataset.
ER
DER++
CLS-ER
After T1
98.8
98.2
98.7
After T2
67.0
92.1
89.2
87.3
89.0
89.5
After T3
54.0
16.9 95.8
82.0
50.0
90.0
78.2
53.5
89.0
After T4
55.9
13.1 22.9
98.7
79.3
33.4
63.0
94.8
81.2
42.4
76.3
87.5
After T5
15.8
15.1 36.0
73.8
98.2
58.4
29.5
67.5
81.1
95.7
69.2
41.5
76.8
83.3
41.1
CCL (SM Default)
CCL (WM)
CCL (IBL)
After T1
98.8
98.3
97.0
After T2
97.0
64.0
92.0
79.7
88.3
77.2
After T3
94.7
72.7
48.5
92.6
57.2
77.0
83.5
43.0
78.5
After T4
91.9
64.5
62.3
65.4
85.7
52.5
48.9
90.5
69.0
35.3
44.9
90.4
After T5
81.1
49.4
59.6
84.8
79.2
54.6
47.3
51.9
84.3
95.8
42.9
33.6
46.6
73.3
93.2
T1
T2
T3
T4
T5
T1
T2
T3
T4
T5
T1
T2
T3
T4
T5
Figure S2: Task probability analysis on Seq- CIFARI0 dataset with 200 buffer size. Semantic mem- ory shows better stability while working model shows better plasticity.
12

---

Under review as a conference paper at ICLR 2023
Table S1: Comparison of different methods on standard CL benchmarks (Class-IL, Task-IL settings),
including non-ER based methods on Seq-CIFARIO 10 dataset
B
Method
Class-IL IL
Task- IL
JOINT
92.2010.15
98.31
SGD
19.62.10.5
61.02.13.3
oEWC
19.12
68.29..3.92
SI
19.48
68.05.15.91
LwF
19.61 #0.05
63.29:22.55
PNN
95.1 95.13
ER
44.79
91.19::0:4
DER++
14.88
91.92:1060
Co
65.57
93.43.10.78
200
ER-ACE
62.08+1144
92.20++057
CLS-ER
66. 19.19
93.90++0.60
CCL
70.04
94.49::0.88
ER
57.74
93.61
DER++
72.70+
93.88:10.50
Co²L
74.26+t0.77
95.90++0.6
500
ER-ACE
68.45
93.47:1100
CLS-ER
75.22:+0.71
94.94+0053
CCL
76.20
95.95 10.14
Table S2: Comparison on Seq-CIFAR1O0 100 dataset for different tasks on 500 buffer size
B
Method
5-Tasks
10-Tasks
20-Tasks
ER
28.02+031
21.49+0.47
16.52=0.86
500
DER++ CCL
41.4040.6
36.20 10.52
22.25
53,23+1662
41.09
33.60 :10.25
B INDUCTIVE BIAS
The shape extraction is performed by applying filter on the input image. Multiple filters were con- sidered (such as Canny (Ding & Goshtasby] (001). Prewitt), but the Sobel filter (Sobel & Feldman 1968) was chosen because produces more realistic output by being precise and also smoothing the edges. The overall algorithm is explained in the following.
Algorithm Sobel Algorithm Shape Extraction
Input: Input data Frgb
1: Up-sample the images to twice the original size: Zrgb Us(girhh)
2: Apply Gaussian smoothing to reduce noisy edges: Tg Gaussian Blur kernel_size 3)
-
+17
--1
-2
-17
3: Get Sobel kernels: S, -2
0
+2
and S,
0
0
-
+1
+2
+1
4: Apply Sobel kernels: 2dtx = Fg * S2 and Bay Fg the 2-dimensional convolution operation
S
5: The edge magnitude: Tahope 2 + x
6: Down-sample to original image size: shaape ds (Ssape))
Figure S3 displays few examples of applying the Sobel operator on the original RGB images. The Sobel output is fed to the IBI. model.
13

---

Under review as a conference paper at ICLR 2023
Input
Sobel
Figure S3: Visual examples of the shape images using Sobel operator
C CCL ALGORITHM
Algorithm 2 Cognitive Continual Learner (CCL)
Input: Dataset De, Buffer B
Initialize: Three networks: Encoder and classifier f parameterized by 0w M. Osas, and lIBL
1: for all tasks E 2.2... do
2: Sample mini-batch: (Ie, ye) ~ Dr
3: Extract shape images: Fc, IB(x.2) where IB is Sobel filter
4:
Csupw CCE( MW, Me)
5:
LsuPIeL LCr(s(x. (ice,;00BBi, 01BL))8.)
6:
if B + then
7:
Sample mini-batch: (06, ybb B
8:
Extract shape images: Eb, = (xx,)
9:
Calculate supervised loss:
10:
CSapwar += [C 0ww)) Vb)
11:
Lsupres += (f
: : (111))))))))) ybb)
12:
Knowledge sharing from semantic memory to working model and inductive bias learner:
13:
LKSW = (x) 0 - f(ms: Bw MI 1
LKSIEL = [ 0sM) f Or.)
15:
Bidirectional knowledge sharing between working model and inductive bias learner: [x; 9w M) (ax 01BL)
LAKS x~D:UB
17:
Calculate total loss:
LWM = LSupwar ALMKS + XLKSWM
19:
CIBL CsupreL +Luks + L'LSS
20:
Update both working model and inductive bias learner: 0w M. 1IBL Stochastically update semantic memory:
21:
22:
Sample s ~ (0,
23:
if then
24:
0SM = d OsM + (1 - d) Owm
25:
Update memory buffer B
return model 0sM
14

---

Under review as a conference paper at ICLR 2023
D SETTING AND DATASETS
We evaluate all methods in different CL settings. Van de Ven & Tolias (2019) describes three differ- ent settings based on increasing difficulty; task incremental learning (Task-IL), domain incremental learning (Domain-IL), and class incremental learning (Class-IL). In Class- -I, each new task consists of novel classes, and the network must learn both new classes while retaining information about the old ones. Task-IL is similar to Class-IL but assumes that the task labels are accessible at both train- ing and inference. In Domain-IL, the classes remain the same for each task, but the distribution varies for each task. We report the results for all three settings on the relevant datasets. CLass-IL is relatively the most complex setting of the three and is widely studied; however, there are some assumptions that simplify this setting to be realistic. Mi et al. (2020) highlighted some of the limi- tations of Class-IL. IL, such as the assumption of the same number of classes across different tasks, no reappearance of classes, and the sample distribution per class well balanced. Hence, Generalized Class-IL (GCIL) was suggested to overcome these limitations and introduce more realistic setting. GCIL is a more generalized CL setting, where the number of classes in each task is not fixed, and the classes can reappear with varying sample sizes. GCIL samples the number of classes and samples from a probabilistic distribution. The two variations are Uniform (fixed uniform sample distribution over all classes) and Longtail (with class imbalance).
We report results on all three settings: Task-IL, Domain- and CLass-IL. Furthermore, we also consider the GCIL setting for one of the dataset additional evaluation setting. All reported results are averaged over three random seeds.
E HYPERPARAMETERS
We utilize a small validation set to tune the hyperparameters for all methods. For Seq-CIFARIO report the results of the original articles (Buzzega et al. 2020 Cha et al. 2021 Caccia et al. 2021 Arani et al., [221I). For the other datasets, in the paper for a similar dataset. For Seq-CIFAR 100 and GCIL-CIFARI00, we formed the search range using the Seq-CIFAR10 10 huperparameters as reference point. Search ranges shown in Table S3
grid search over the hyperparameters reported
Do dataset is more complex compared to the CIFAR versions and includes images of larger sizes. Hence, we consider the Seq-Tinylmagenet hyperparameters in the respective paper as the reference point for further tuning. The learning rate ir, the number of epochs, and the batch size are similar across the datasets. The ema update rate is lower for more complex datasets, as shown in CLS-ER. r chosen in the range of [0.01, 0.1] with step size of 0.02 for CLS-ER and CCL. The different hyperparameters chosen for the baselines, after tuning, are shown in Table S4
The different hyperparameters chosen for CCL are shown in Table S5 size, number of epochs are uniform across all datasets. The stochastic update rate and decay pa- rameter are similar to CLS-ER. The hyperparaneters and stable across settings and datasets and and also compliment each other. The loss balancing weights are reported as four different parameters for clarity, however, they show similar pattern. Therefore, CCL does not require extensive fine-tuning across different datasets and settings.
The parameters Ir, batch
F COMPLEXITY
We discuss the computational complexity aspect of our proposed method. CCL involves three net- works during training; however, in inference, only single network is used (SM module). Therefore, for inference purposes, the MAC count, the number of parameters, and computational capacity re- main the same as the other single-network methods.
The training cost requires three forward passes, as it consists of three different modules. ER, DER++, CO and ER-ACE have a single network. CLS-ER also has three networks and therefore requires 3 forward passes. CCL has training complexity similar to CLS-ER; however, it outperforms CLS-ER in all provided metrics.
15

---

Under review as a conference paper at ICLR 2023
Table S3: Search ranges for tuning hyperparameters
Method
Hyperparameters
Search Range
ER
br
[0.01, 0.03, 0.1, 0.5]
1r
[0.01, 0,03, 0.1]
DER+
[0.1, 0.2, 0.5]
[0.5, 1.0]
Ir
[0.01, 0.03, 0.1]
[0.01, 0.1,
CO
k
[0.2, 0.5]
k-
[0.01, 0.05]
[100, 150]
ER-ACE
lr
[0.01, 0.03, 0.1, 0.5]
lr
[0.01, 0.03, 0.1]
10.1, 0.3]
CLS-ER
Tp
[0:1:0.1]
[0:1:0.1]
Op
[0.99,0.999]
[0.99, 0.999]
Ir
[0.01, 0.03, 0.1]
r
[0:1:0.1]
[0.99,0.999]
CCL
[0.01, 0.1]
[0.01, 0.1]
X
[0.01, 0.1]
[0.01, 0.1]
On the memory front, similar to all methods, memory samples based the memory budget allotted (200 and 500 in the experiments). There are no additional memory requirements, as we do save any extra information (such logits) to be used later in our objectives.
G OTHER METRICS
Forward transfer, backward transfer, and forgetting are other metrics (Lopez-Paz & Ranzato 2017 used in CL literature. These metrics are estimated from the model checkpoint after a task is com- pleted, this checkpoint has the highest accuracy for that particular task. However, this does not hold true for our method, which utilizes the stochastically updated model for inference and eval- uation purposes. The SM module assimilates knowledge from the working model and is updated stochastically by the exponential moving average. It achieves highest accuracy on previous tasks while also learning the new tasks. Therefore, the results may be misleading.
However, we evaluate backward transfer (-1 x forgetting) by considering the best accuracy of the SM module after a particular task and then finding the difference between this and the final accuracy. Taking into account Figure [22 if we use the backward metric formula directly, we get positive backward transfer for CCL in Tasks 3 and 4 (due to the SM model achieving high accuracy in the previous task while also learning the task that stochastic update frequency); therefore, pick the maximum one and subtract it from the last row. We report the values for the metrics in Table S7
CCL fares better in backward transfer (or forgetting) compared to other techniques.
H EXTENDED RELATED WORKS
One of the modules (IBL) in CCL utilizes the inductive bias in terms of shape to produce more generic representations. There are several works Geirhos et al. (2018) that showcase the texture bias problem of neural networks. Several techniques have been introduced to reduce texture bias
16

---

Under review as a conference paper at ICLR 2023
Table S4: Selected hyperparameters for all baselines.
Dataset
[8]
Method
Hyperparameters
ER
1r=0.1
DER+
Ir=0.03, a==0,0 10=00,,,0000
200
CO'L
1r:0.5, r:0.5, k:0.2, x e:100
e:100
ER-ACE
1r=0.01
CLS-ER
Or= A=0.15, 0,000 a,,00.99, a,=0.999
Seq. 100
ER
1r=0.1
DER++
1r=0.03, a= 3=0.5
500
CO
Ir:0.5, r:0.5, 6:0.2, x² x=:0001, e:100
ER-ACE
1r=0.01
CLS-ER
(r=0.1 A=0.15, 1, r,
a, a,=0.999, a,90.999
ER
1r=0.1
DER+
[r=0.03, 0=0.5, 3=0.1
200
CO
ER-ACE
Ir=0.1
CLS-ER
lr=0.1 A=0.1, r,,7,,,,,0.7 000 a,=00999,
CILLCCIAROO0
ER
1r=0.1
+ER++
1r=0.03, o=0.2, 3=011
500
CO'L
ER-ACE
(r=0.1 1.0 a,=00999 a,=0999
CLS-ER
1r=0.1
ER
1r=0.1
200
DER+
1r00, a00.1, 3=1.0
CLS-ER
(r=0.05 0,00 a,
0,=0.999
Domain
ER
[r=0.1
500
DER++
Ir=0.03, 0=0.5, 3=0.1
CLS-ER
1==0.05 X=0.1, r,00088 r,00005, 0,=0.999. a,=00999
Table S5: Selected hyperparameters for CCL across different settings.
batch size
[B
Ir
#epochs
d
X
?
Scq-CIFAR1O
200
0.03
32
0.2
0.999
0.1
0.1
0.1
0.1
500
0.03
32
50
0.2
0.999
0.1
0.1
0.1
0.1
Seq-CIFARI00
200
0.03
32
50
0.06
0.999
0.1
0.01
0,1
0.01
500
0.03
32
50
0.08
0.999
0.1
0.01
0.1
0.01
200
0.03
32
50
0.09
0.999
0.1
0.01
0.1
0.01
500
0.03
32
0.2
0.999
0.1
0.01
0.1
0.01
DN4IL
200
0.03
32
50
0.06
0.999
0.1
0.01
0.1
0.01
500
0.03
32
50
0.08
0.999
0.1
0.01
0.1
0.01
Table S6: Fifteen different natural corruptions
Gaussian Noise, Impulse Noise, Shot noise, Speckle noise Defocus blur, Glass blur, Motion blur, Zoom blur, Gaussian blur Brightnesss Contrast, Fog, Frost, Snow
Corruptions
Elastic Transformation, JPEG compression, Pixelate, Spatter, Saturate
and improve representations. Geirhos et al. 2018 increases shape bias by adding multiple stylized images along with the original images used for training. Styles of artistic paintings transferred
17

---

Under review as a conference paper at ICLR 2023
Table S7: Backward transfer metric on Scq-CIFARI0 dataset
Method
Backward Transfer
ER
-61.75
DER+
-33.45
CLS-ER
-23.47
CCL
-6.07
Table S8: Accuracy on the proposed DN4IL dataset for the Domain-IL setting, CCL shows signif- icant improvement in all disparate and challenging domains.
Method
clipart
infograph
painting
sketch
quickdraw
Acc
JOINT
59.93
SGD
9.98+054
19.97
2.32
6.58
14.91
71.23
20.83
20.08:40.5
26.37::0.5
5.56
13,92=0991
23.69+1.54
69.95
26.59:t0.31
DER++
33.66:1.65
37.24:1064
9.80
24. 1.11
34.37:12.00
69.26
34.75
200
CL.SERR
45.53:
49.17:1.12
15.79=048
35.80=0.64
48.0332005
54, 14.40=1.25
40.83: 1.07
CCL
47.524025
54.69
15.70=0.33
37.54
51.98
58, 0011
44.23
27.54:4006
31,89
#.89
19.39=1.02 1.02
28.36
70.96=0010
31.01
DER++
14.49:1..9
46.17:10.5
14,01
33.44=0.90
43.59=1111
69,53
41.87::0.63
500
CLS-ER
49.85+0088
51.41
18.17=0008
37.94=0.44
49.02+1.57
55.63=0071
43.41 =0.800
CCL
54.77.1.15
60.37.0075
19.35
=4.50
56.34
11.73
49.32.10.23
to Imagenet dataset to create stylization (ADaIN) and is an additional offline process. Chen et al. (2016) uses generative tech- niques (InfoGan) to synthesize images that are less biased to texture. Li et al. (2020) also creates an augmented dataset by blending the texture of one image and shape of another image in the training set to create a new image. All of these techniques, then merge both the original and synthesized data to train the network on a bigger dataset. However, training single network with these different dis- tributions leads to learning sub-optimal representations. This is also shown in the results in Geirhos al. (2018), where they had do an additional fine-tuning the original dataset achieve better results on the original data. Also, synthesizing and generating new data is expensive and might come with unaccounted bias.
Style-transfer is performed using adaptive instance
CCL on the other hand, tries to leverage on the under-utilized implicit shape information with min- imal overhead. There is requirement of additional data, generative networks and the RGB and shape data are not combined together to make one big training dataset. The RGB image is fed to one network and the shape information is learnt by another network (IBL) and the supervision is provided via knowledge transfer between these two networks and is mutual. Each network has enough flexibility to learn on its own feature while also getting guidance from the other feature.
I DN4IL
We introduce new dataset for the Domain-IL. setting. is subset of the standard DomainNet dataset (Peng et al. 2019) used in domain adaptation. It consists of six different domains real, clipart, infograph, painting, quickdraw, and sketch. The shift in distribution between domains is challenging. Few examples can be seen in Figure S4
Each domain includes 345 classes, and the overall dataset consists of 550000 samples. The classes have redundancy, and also evaluating the whole dataset can be computationally expensive for CL settings. Therefore, we create a subset by grouping semantically similar classes into 20 super cate- gories (considering class overlap between other standard datasets Each super category has five classes each, which results in a total 100 classes. The specifications of the classes are given in Table (59 The dataset consists of 67080 training images and 19464 test images. The image size for all experiments is chosen as 64 x 64 (the normalize transform is not applied in the augmentations).
also facilitate OOD analysis).
18

---

Under review as a conference paper at ICLR 2023
real
clipart
Infograph
painting
sketch
quickdraw
airplane
flavours
cream
TOCO
Wine
Pinot Noir
Figure S4: Visual examples of DN4IL dataset
Table S9; Details supercategory and classes in DN4IL dataset.
supercategory
class
small animals
mouse tiger
squirrel bear
rabbit lion
dog
raccoon
medium animals
panda zebrä
large animals
camel horse
kangaroo
clephant cow
aquatic mammals
whale shark
dolphin octopus
non-insect invertebrates
snail
scorpion
spider
lobster crab
insects
bec
butterfly
mosquito
bird but
vehicle
bus
bicycle
motorbike
train pickup bruck
sky-vehicle
airplane
flying_samn sauuer
aircraft_carrier
helicopter apple
_n_air_halloo
fruits
strawberry
banana
watermelon
10
vegetables
carrot
asparagus
mushroom
onion broccoli
11
music
trombone
violin cello
guitar
clarinct bed
12
furniture
chair
dresser table
couch
13
household electrical devices tools
clock
floor_lamp
telephone
television
keyboard
14
saw
axe
hammer
screwdriver
scissors
15
clothes accessorics
bowtic pants jacket sock
shorts
16
man-man man- outdoor
skyscraper windmill house
castle
bridge
17
cloud bush birthday-cake hamburger icc.cream calendar marker map
mountain
18
food
sandwich
pizza
19
stationary
pencil
20
household items
wite_bottle cup teapot
frying-pan
wine_glass
19

---

Under review as a conference paper at ICLR 2023
12000
10000
Samples
8000
6000
4000
2000
real
clipart infograph painting sketch quickdraw Domains
Figure S5: Number samples per domain in DN4IL dataset.
3500
3000
2500
Samples
2000
1500
1000
6
7
11
12
13
14
17
18
19
20
Super Categories
Figure S6: Number of samples per super category DN4IL dataset.
Samples
600
400
200
butt Moree
Hot SARSS
raig
pickup
water
Classes
Figure S7: Number of overall samples per class in DN4IL dataset.
20

---

Under review as a conference paper at ICLR 2023
real
Samples
502
clipart
602
Samples
102
infograph
200
painting
202
sketch
602
Samples
302
quickdraw
500
Samples
400
300
200
16
Figure S8: Number of samples per supercategory for each domain in DN4IL dataset.
21


Reviews:
[{"review_id": "uTHmRJai4d", "reviewer": "Reviewer_6gTb", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a method for continual learning (Cognitive Continual Learner, CCL) with three components: a ‘working model’ that learns to classify from raw input, an ‘inductive bias learner’ that learns to classify from a processed version of the input in the form of shape information, and a ’semantic learner’, which is a stochastic moving average of the parameters of the working model. Apart from the classification losses on the first two modules, there are also few auxiliary ‘knowledge sharing’ objectives: a bidirectional MSE loss between the outputs of the working model and the inductive bias learner, enforcing them to make consistent predictions, and MSE losses between the output of the semantic learner and each of the working model and inductive bias learner, constraining each of them to not veer too far from historical predictions. The model also uses a replay buffer maintained with reservoir sampling. Experiments in both the task incremental and class incremental settings are run on Sequential CIFAR-10, Sequential CIFAR-100, GCIL-CIFAR100 (which blurs boundaries between tasks) and a custom dataset introduced in the paper called Domain2IL, which is a subset of the existing DomainNet dataset, curated to make the class distributions more balanced and uniformly distributed. CCL is compared to a number of replay-based baselines and is shown to be the best performing across all tasks. Some additional analyses are run to demonstrate the robustness of the method, how it fairs with the plasticity-stability tradeoff, and some ablations of the components of the model.", "summary_of_the_review": "Update: I have increased my score to 6 after authors responded to my concerns.\n------\nThis paper presents an interesting method for continual learning, that features enforcing consistency between multiple modules, that is justified by strong experimental results on a number of CL visual classification benchmarks comparing to a number of relevant replay-based baselines. In its current form, the paper is lacking a number of important implementational details that render it entirely unreproducible (described in more detail above), which is the main reason I think it cannot be accepted as is. It is also difficult to tell from the ablations which components of model architecture and loss functions are most important to performance, as all the ablations do well - I think further ablations are needed in order to determine this (as descibed above), which would provide a lot more insight into why the model works well. I would be willing to increase my score if these two points were addressed.", "strength_and_weaknesses": "- The method in the paper has a number of interesting features:\n    - (i) the knowledge sharing between multiple modules. Knowledge sharing via distillation between two modules (that usually evolve at different timescales) has been used in continual learning in a number of existing methods (e.g. Progress and Compress, Schwarz et al 2018), but CCL enforces consistency between 3 modules.\n    - (ii) enforcing consistency between two different views of the data (the raw input and the shape information)\n- The empirical results are strong, and are run on relevant benchmarks comparing against suitable baselines.\n\nOpportunities for improvement / questions\n- There are very important details missing about the implementation that make it difficult to fully understand the method, and certainly impossible to reproduce (given that no code is provided).\n    - (i) There seems to be no explanation of how the shape information is extracted from the data and represented to the inductive bias learner. This appears to be a key feature to the model. Indeed in 5.3 it is even stated that the primary benefit of CCL comes from the shape information.\n    - (ii) The stochastic momentum update does not seem to be described - what is the nature of the stochasticity in this update and how important is it to the performance of the model?\n- In the ablations it would be interesting to see what would break the model - all the ablations seem to fair quite well, for instance much better than the experience replay baseline, so it is difficult to understand which aspects of the model are the most important. Maybe some further ablations are needed to ascertain this, e.g. how would the working model perform by itself without either the SM or the IBL? Would it be correct to say that this should be similar to the performance of ER?\n- It is mentioned that some of the results in Table 1 are taken from the original papers but some are run again by the authors - but it is not clear which are rerun and which are not.\n- The plasticity and stability metrics used are not obviously reflective of the conventional meaning of the terms. Stability is computed as the average accuracy all tasks 1:T-1 after learning of the final task T, but this metric is highly dependent on the initial performance on tasks 1:T-1 - would it not be more informative to report a more standard metric like forgetting or backward transfer (Lopez-Paz et al. 2017)? Also, why do the plasticity and stability values in Figure 2 go above 100%?\n- What is the rationale for using an MSE loss for the Knowledge-sharing loss rather than say a Kullback-Leibler loss, when the outputs of the WM and IBL networks are probability vectors?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "kBjPNSNAcs", "reviewer": "Reviewer_J9Rf", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes Cognitive Continual Learner (CCL) which is composed of three modules. An explicit module that learns from the input and two implicit modules (inductive biases and semantic memories) that share indirect contextual knowledge. CCL is evaluated under a number of continual learning settings, including on a novel benchmark, and compared with recent experience replay continual learning methods, demonstrating favourable performance.", "summary_of_the_review": "The proposed system is interesting combining a few successful elements from recent literature but empirical evaluations (metrics, baselines) require improvement.", "strength_and_weaknesses": "Strengths:\n+ The proposed system is simple combing several elements from recent work but outperforms all baselines.\n+ Extensive evaluations are conducted under numerous continual learning settings.\n\nWeakness:\n- CCL was compared with a number of recent baselines but the comparison only focused on classification performance disregarding complexity. How does CCL compare with these methods in terms of memory, complexity, etc?\n- Common continual learning metrics such as forward and backward knowledge transfer are not reported.\n- No comparison to non experience replay methods.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Yfgokpkf5WQ", "reviewer": "Reviewer_e4pE", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper draws inspiration from the cognitive science literature to design a multi-module architecture for continual learning from images. More specifically, the proposed “Cognitive Continual Learning” consists of a working model (which processes images in the typical way), an inductive bias learner (which receives pre-processed shape information) and a semantic memory (which is a delayed version of the working model), whereby the predictions of the different models are encouraged to be similar to each other by various additional loss terms. In extensive empirical comparisons the authors show that the resulting model performs better than several established continual learning methods (e.g., DER++, Co2L and CLS-ER) on variants of task-, domain- and class-incremental learning.\n", "summary_of_the_review": "Although I find the proposed multi-module architecture interesting, I believe the provided empirical comparisons are not fair. There is also an important issue regarding the paper’s description of continual learning settings.\n", "strength_and_weaknesses": "The writing and organization of the paper are generally clear, and I find the proposed multi-module architecture an interesting contribution. The Domain2-IL dataset protocol that is constructed could also be a useful contribution (although I’d recommend the authors to reconsider the name).\n\nThe paper however has some important issues that require addressing.\n\nFirst, the fairness of the empirical comparisons. The main empirical component of the paper is comparisons against methods such as ER, DER++ or CLS-ER. However, an important confound in this comparison is that the proposed CCL method uses shape information while the other methods do not. That is, the proposed CCL method uses additional information (or you could see it as additional pre-processing or augmentations) that is not available to the other methods, making these comparisons unfair.\nRather than including these comparisons and trying to make a claim of “state-of-the-art”, I would recommend the authors to instead focus on carefully and systematically characterizing the performance of their proposed methods and its different components (e.g., along the lines of the ablation experiments in Table 2, but more extensively).\n\nAnother important issue is the way this paper describes the “continual learning settings”, and its claim that the proposed CCL shows improvement “across all continual learning settings”. By “all continual learning settings” this paper seems to mean Task-IL, Domain-IL, Class-IL and General Class-IL. The distinction between Task-IL, Domain-IL and Class-IL seems based on this paper (https://arxiv.org/abs/1904.07734, although the paper is not discussed or cited, which it probably should be). With regards to these three scenarios I could agree with referring to them as “all settings” (because these are the three ways in which a given sequence of tasks can be performed), but the addition of General Class-IL complicates things. General Class-IL is not just about the way a sequence of tasks is performed, but additionally has restrictions about how that sequence is constructed (e.g., gradual shifts between tasks). When also considering the way a task sequence is constructed, there can be many more settings than just the ones considered in this paper.\n\nIt is of course fine to consider a continual learning set up as described by General Class-IL, but it is important not to contrast it (or put it at the same level of abstraction) with Task-IL, Domain-IL and Class-IL. Rather, General Class-IL is a subset of Class-IL, and I think it is important to be clear about that.\n\nAs a final note, it might be good to reconsider the name of the proposed Domain2-IL protocol. Currently this name has the risk of suggesting that it is a modified version of the Domain-IL scenario, while it is an instantiation of the Domain-IL scenario (i.e., similar to for example permuted MNIST, but – as the authors rightfully point out – a more realistic one).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "UuPdCnxk8q", "reviewer": "Reviewer_BtBz", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new continual learning method inspired by human abilities to continually learn with no catastrophic forgetting. Based on aspects in cognitive architecture, 3 modules are combines in the proposed solution.  The authors suggest that the shape of objects is not well leveraged in current CNNs and use a specific module for learning from shapes. The third module acts as a memory module and is updated via momentum updates. Experiments on cifar10 and cifar100 show improvements over compared methods. Also a domain incremental sequence is proposed based on DomainNet benchmark.\n ", "summary_of_the_review": "A  new cognitive inspired solution to continual learning is proposed with multiple networks. Some aspects are missing and comparisons are limited. To this reviewer, the paper int its current shape is not robust enough.", "strength_and_weaknesses": "+ New brain inspired solution to continual learning.\n+ Good results.\n- A part from the cognitive emulation, there doesn't seem good reasons behind the chosen design strategies.\n- 3 networks are used increasing the computational footprint of the proposed  solution. Computational aspect is not discussed. A  fair comparison with other methods would be by increasing the buffer size to be equal to the amount of storage due to the extra networks.\n- The method requires large number of hyperparameters.\n- Not sure why the new domain incremental sequence is introduced. Core50 domain incremental learning sequence could be used. \n- Only cifar based sequences are considered, what about ImageNet, TinyImageNet or Core50?\n- Not sure why the authors picked only 5 tasks sequence, usually evaluation is done with different number of classes per task in order to show the robustness of the method. \n- Not sure that the method is state of the art, there seem to be new published works that show better performance, e.g., PoDNet[1]\n- The use of the shape limits the applicability of the method on recognizing objects with discriminating characteristics other than shape, e.g., a horse and a zerbra.\n[1] Ashok, Arjun, K. J. Joseph, and Vineeth Balasubramanian. \"Class-Incremental Learning with Cross-Space Clustering and Controlled Transfer.\" ECCV 2022.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject