A Study of Biologically Plausible Neural Network: The Role and Interactions of Brain-Inspired Mechanisms in Continual Learning
Authors: Fahad Sarfraz
Elahe Arani
Bahram Zonooz
Venue: Accepted by TMLR ()

Abstract: Humans excel at continually acquiring, consolidating, and retaining information from an ever-changing environment, whereas artificial neural networks (ANNs) exhibit catastrophic forgetting. There are considerable differences in the complexity of synapses, the processing of information, and the learning mechanisms in biological neural networks and their artificial counterparts, which may explain the mismatch in performance. We consider a biologically plausible framework that constitutes separate populations of exclusively excitatory and inhibitory neurons that adhere to Dale's principle, and the excitatory pyramidal neurons are augmented with dendritic-like structures for context-dependent processing of stimuli. We then conduct a comprehensive study on the role and interactions of different mechanisms inspired by the brain, including sparse non-overlapping representations, Hebbian learning, synaptic consolidation, and replay of past activations that accompanied the learning event. Our study suggests that the employing of multiple complementary mechanisms in a biologically plausible architecture, similar to the brain, may be effective in enabling continual learning in ANNs. \footnote{We will make the code available upon acceptance.

Published in Transactions on Machine Learning Research (04/2023)
A Study of Biologically Plausible Neural Network: The Role and Interactions of Brain-Inspired Mechanisms in Continual Learning
Fahad Sarfrazi1,, Elahe Aranitl2, Bahram Zonooz 1.2
Jahad. sur furaraa @navinfo.cu. {e. arans, bahram, zonooz) @ggmail com Advenced Research Lab, Navinfo Europe, Eindhoven, The Netherlands
Department of Mathematics and Computer Science, Eindhoven University of Technology, The Netherlonds *Contributed equally.
Reviewed on OpenReview: https: / openreview. pet/forunts id= Jd-
Abstract
Humans excel at continually acquiring, consolidating, and retaining information from an ever-changing environment, whereas artificial neural networks (ANNs) exhibit catastrophic forgetting. There are considerable differences in the complexity of synapses, the processing of information, and the learning mechanisms in biological neural networks and their artificial counterparts, which may explain the mismatch in performance. We consider a biologically plausible framework that constitutes separate populations of exclusively excitatory and in- hibitory neurons that adhere Dale's principle, and the excitatory pyramidal neurons are augmented with dendritic-like structures for context-dependent processing of stimuli. We then conduct comprehensive study the role and interactions of different mechanisms inspired by the brain, including sparse non-overlapping representations, Hebbian learning, synaptic consolidation, and replay of past activations that accompanied the learning event. Our study suggests that the employing of multiple complementary mechanisms in a bio- logically plausible architecture, similar to the brain, may be effective in enabling contimual learning in ANNs.
1 Introduction
The human brain excels at inually learning from dynamically changing environment, whereas standard artificial neural networks (ANNs) are inherently designed for training from stationary i.i.d. data. Sequential learning of tasks in continual learning (CL) violates this strong assumption, resulting catastrophic forget- ting. Although ANNs are inspired by biological neurons (Fukushima) 1980), they omit numerous details of the design principles and learning mechanisms in the brain. These fundamental differences may account. for the mismatch in performance and behavior.
Biological neural networks are characterized by considerably more complex synapses and dynamic context- dependent processing of information. In addition, individual neurons have a specific role. Fach presynaptic neuron has an exclusive excitatory or inhibitory impact on its postsynaptic partners, as postulated by Dale's principle (Strata al.) 1999). Furthermore, distal dendritic segments in pyramidal neurons, which comprise the majority of excitatory cells in the neocortex, receive additional context information and enable context-dependent processing of information. This, in conjunction with inhibition, allows the network to learn lask-specific palterns and avoid catastrophic forgetting (Yang et al.) 2014 [yer et. al., 2022 Barron et al. 2017). Furthermore, replay of non-overlapping and sparse neural activities of previous experiences in the neocortex and hippocampus is considered to play a critical role in memory formation, consolidation, and retrieval WWalker & Stickgold 2004 McClelland et al. 1995). To protect information from crasure,
We will make the code available upon acceptance.

---

Published in Transactions on Machine Learning Research (04/2023)
Context
(a)
h/
1-11
We
Wa
Web
Wer
Dendrite Segments
(b)
inactive pyramidal cell
active pyramidal cell
inhibitory cell
W
Dropout
k-WTA
fully connected
Hebbian
i
Update
context vector leamable weights
Wie
Figure 1: Architecture of hidden layer in the biologically plausible framework. Each layer consists of separate populations of exclusively excitatory pyramidal cells and inhibitory neurons, which adhere to Dale's principle. The shade indicates the strength of weights or activations, with darker shade indicating higher value. (a) The pyramidal cells are augmented with dendritic segments, which receive an additional context signal e and the dendritic segment whose weights are most aligned with the context vector (bottom row) is selected to modulate the output aclivity of the leedforward neurons for context-deeendent processing of information. (b) The Hebbian update step further strengthens the association between the context and the winning dendritic segment with maximum absolute value (indicated with darker shade in the bottom row). Finally, Heterogeneous dropout keeps the activation count of each pyramidal cell (indicated with the gray shade) and drops the neurons that were most active for the previous task (the darkest shade dropped) to enforce non-overlapping representations. The top-k remaining cells then project to the next layer (increased shade). This provides us with a more biologically plausible framework within which we can study the role of different. brain-inspired mechanisms and provide insights for designing new CL methods.
the brain employs synaptic consolidation, in which plasticity rates are selectively roduced proportion strengthened synapses (Cichon & Gan 2015).
Thus, study the role and interactions of different mechanisms inspired by the brain in biologically plausi- ble framework in CL setup. The underlying model constitutes separate populations of exclusively excitatory and inhibitory neurons in each layer, which adheres to Dale's principle (Cornford et al. 2020 and excitalory neurons (mimicking pyramidal cells) are augmented with dendrite-like structures for context-dependent pro- cessing of information (Iyer et al. 2022).. Dendritic segments process an additional context signal encoding task information and subsequently modulate the foedforward activity of the excitatory neuron (Figure I We then systematically study the effect of controlling the overlap in representations, employing the *fire to- gether, wire together" learning paradigm and employing experience replay and synaptic consolidation. Our study shows that:
i. An ANN architecture equipped with context-dependent processing of information by dendrites and adhering to Dale's principle can learn effectively in CL setup. Importantly, accounting for the discrep- ancy in the effect of weight changes in excitatory and inhibitory neurons further reduces forgetting in CT.
Enforcing different levels activation sparsity in the hidden layers using k- winner-take-all activations and employing complementary dropout mechanism (Heterogeneous Dropout) that encourages the model to use a different set of active neurons for each task can effectively control the overlap in representations, and hence reduce interference while allowing for resusability.
2

---

Published in Transactions on Machine Learning Research (04/2023)
ii. Task similarities need to be considered when enforcing such constraints to allow for balance between forwarding transfer and interference.
iv. Mimicking the ubiquitous "fire together, wire together' learning rule in the brain through Hebbian update step the connection between context signal and dendritic segments, which further strength- ens context gating and facilitates the formation of task-speeific subnetworks.
v. We show that employing both synaptic consolidation with importance measures adjusted to take into account the discrepancy in the effect of weight changes and replay mechanism in context-specifie manner critical for consolidating information across different tasks, especially for challenging CL settings.
Our study suggests that employing multiple complementary mechanisms in a biologically plausible architec- ture, similar to what is believed to exist in the brain, can be effective in enabling CL in ANNs. To the best of our knowlodge, are the first to provide a comprehensive study of the integration of different brain-inspired mechanisms in biologically plausible architecture in CL setting.
2
Biologically Plausible Framework for CL
We provide details of the biologically plausible framework within which we conduct our study.
2.1 Dale's Principle
Biological neural networks differ from their artificial counterparts in the complexity of synapses and the role of individual units. In particular, the majority of neurons in the brain adhere to Dale's principle, which posits that presynaptic neurons can only have an exclusive excitatory or inhibitory impact on their postsynaptic
partners (Strala et al. 1999). Several studies show that the balanced dynamics (Murphy & Miller 2009
Van Vreeswijk & Sompolinsky 1996) of excitatory and inhibitory populations provide functional advantages,
including efficient predictive coding (Boerlin et al., 2013) and pattern learning (Ingrosso & Abbott 2019 Furthermore, inhibition hypothesized
play a role in alleviating catastrophic forgetting (Barron et al. 2017). Standard ANNs, however, lack adherence to Dale's principle, as neurons contain both positive and negative output weights, and signs can change while learning.
Cornford et al. [0200 incorporate Dale's principle into ANNs (referred to as DANNs), which take into account the distinct connectivity patterns of excitatory and inhibitory neurons (Tremlay et al. 2016) and perform comparable to standard ANNSs in the benchmark object recognition task. Each layer I comprises of a separate population of excitatory, h, E R24 and inhibitory in E RR neurons, where ne >> ni and synaptic weights are strictly non-negative. Similar to biological networks, while both populations receive excitatory projections from the previous layer (h!-1), only excitatory neurons project between layers, whercas inhibitory neurons inhibit the activity of excitatory units of the same layer. Cornford et al. 2020) characterized these properties by three sets of strictly positive weights: excitatory connections between layers W E I="'"", excitatory projection to inhibitory units W!. R*xx*, and inhibitory projections within the layer W R* The output of the excitatory units is impacted by the subtractive inhibition from the inhibitory units:
(W! cc W W MM + b'
(1)
where b' E Re is the bias term Figure shows the interactions and connectivity between excitatory pyramidal colls (triangle symbol) and inhibitory neurons (denoted by t).
We aim to employ DANNs as fowdfr neurons to show that they can also learn in challenging CT. setting and performance comparable to standard ANNs and provide a biologically plausible framework for further studying the role of inhibition in alleviating catastrophic forgetting.
2.2 Active Dendrites
The brain employs specific structures and mechanisms for context-dependent processing and routing of information. The prefrontal cortex, which plays an important role in cognitive control (iileer & Cohen
3

---

Published in Transactions on Machine Learning Research (04/2023)
2001 receives sensory imputs as well as contextual information, which allows it to choose the most relevant sensory features for the present task to guide actions (Mante et al. 2013 Fuster 2015 Siegel et al. 2015 Zeng et al. 2019). Of particular interest are pyramidal cells, which represent the most populous members of the excitatory family of neurons in the brain (Bekkers) 2011). The dendritic spines in pyramid cells cxhibit highly non-linear integrative properties that are considered important for learning Lask-specific patterns (Yang el al. 2014). Pyramidal cells integrate a range of diverse inputs into multiple independent dendritic segments, allowing contextual inputs in active dendrites modulate the response of neuron, making it more likely to fire. However, standard ANNs are based on point neuron model (Lapique 1907) which is an oversimplified model of biological computations and lacks the sophisticated non-linear and context-dependent behavior of pyramidal cells.
Iyer et al. (202)) model these integrative properties of dendrites by augmenting each neuron with a set of dendritic segments. Multiple dendritic segments receive additional contextual information, which is processod using a separate set of weights. The resultant dendritic output modulates the feodforward activation, which is computed by a linearly weighted sum of the feedforward inpuls. This computation results in neuron where the magnitude of the response to given stimulus is highly context-dependent. To enable task-specific processing of information, the prototype vector for task 7 is evaluated by taking the clement- mean of the tasks samples, D, at the beginning of the task and then subsequently provided as context during training.
C
1
Σ FED,
(2)
Dr
During inference, the closest protolype vector to each test sample, x', is selected as the context using the Euclidean distance among all task prototypes, C, stored in memory.
= arg min x - -Cill2
(3)
Following Tyer et al. (2022). we augment the excitatory units in cach layer with dendritic segments (Figure (a)). The leedforward activity of excitatory units is modulated by dendritic segments, which receive context vector. Given the context vector, each dendritic segment j computes uy c, given weight uj E R and the context vector e E Rd where d is the dimensions of the input image. For excitatory neurons, the dendritic segment with the highest response the context (maximum absolute value with the sign retained) is selected to modulate output activity.
h = k- WTA(# x (l))))
where K arg max Ju cl
(4)
i
where a is the sigmoid function Hau & Moraga 1995 and k- WTA(-) is the k-Winner-Takes All activation function (Ahmad & Scheinkman 2019) which propagates only the top k neurons and sets the rest at zero. This provides us with a biologically plausible framework where, similar to biological networks, the feedforward neurons adhere to Dale's principle, and the excitatory neurons mimic the integrative propertics of active dendrties for context-dependent processing of stimuli.
3 Continual Learning Settings
To study the role of different components inspired by the brain in a biologically plausible NN for CL and gauge their roles in the performance and characteristics of the model, we conduct all our experiments under uniform settings. Implementation details and experimental setup the models on two CL scenarios. Domain incremental learning (Domain-IL) refers the CL scenario in which the classes remain the same in subsequent tasks but the input distribution changes. We consider Roi-MNIST which involves classifying the 10 digits in each task with each digit rolated by an angle between 0 and 180 degrees, and ENIM which applies a fixed random permutation to the pixels for each task. Importantly, there are different variants of Rot-MNIST with varying difficultics. We incrementally rotate the digits a fixed degree, i.c. (0, 8, .... (N-1)*8) for task (Tı, (21,7,,.,, T2, TN} which is substantially more challenging than random sampling rotations. Importantly, the Rot-MNIST dataset captures the notion of
provided in Appendix. We evaluate
4

---

Published in Transactions on Machine Learning Research (04/2023)
Table 1: Effect of each component of the biologically plausible framework on different datasets with varying number of tasks. We first show the effect of utilizing feedforward neurons adhering to Dalc's principle in conjunction with Active Dendriles to form the framework within which we evaluate the individual effect of brain-inspired mechanisms (Hebbian Update, Heterogeneous Dropout (HD), Synaptic Consolidation (SC), Experience Replay (ER) and Consistency Regularization (CR)) before combining them all together to forge Bio-ANN. For all experiments, we set the percentage of active neurons at 5. We provide the average task performance and I sid of five runs. We also demonstrate performance gains with the brain-inspired mechanisms on top of standard ANNs in Table 6 in Appendix
Rot-MNIST
Perm-MNIST
Method
Seq-MNIST
5 Tasks
10 Tasks
20 Tasks
5 Tasks
10 Tasks
20 tasks
Joint SGI)
98. 28.277
98. 08.2520..0
98.17
97.64 19.
97.6 97.61=a.1
97.5920.08
94.5320.60
93. 13.79
14.13
51.80
77.96
76.42.2.44
68.1812.1
19.83
Active Dendrites
92.5810.26
71.06
48.1810.52
95.71
24.4
91.74.0.44
19.97
+ Dale's Principle
92.28
70.78+0.2
48. 48.79
96.18
95.20=0.11
92.44+0.20
19.79
+ Iebbiia Update
92.58
71.22+0
48.88
95.90+0.24
94.72
92.55
19.88
+ HD
93. 28.25
75.5020.74
51,11am. an.70
96.58
95. 25.94
93. 20.20
38, 38.45a2.1
+ SC
93.34 00.37
75.94+1.16
64.99
96.54.n.29
96.14 142447
95.4320.4p
27.31
+ ER
95.21 10.28
90.99
96.72 10.33
96.04
94.26 1o.75
86.93...a
F ER + CR
96.48
93.87a0.25
89.39
97.23 2...0
96.93
96. 20.00
89.602o.71
BiN
96.82=o.14
94.64 22
91.32 20
97.33
97.07=0.0s
96.51
89.90
similarity in subsequent tasks, where the similarity between two tasks is defined by the difference in their degree of rotation, whereas each task in Perm-MNIST is independent. We also consider the challenging Class incremental learning (Class-IL) sccnario where new classes are added with each subsoquent task and the agent must learn distinguish not only amongst the classes within the current task but also across all learned tasks. Seq-MNIST divides the MNIST classification into 5 tasks with 2 classes for each task.
4 Empirical Evaluation
To investigate the impact of the different components inspired by the brain, we use the aforementioned biologically plausible framework and study the effect on the performance and characteristics of the model.
4.1 Effect of Inhibitory Neurons
We first study whether feedforward networks with separate populations of excitatory and inhibitory units work well in the CL setting. Importantly, we note that when learning sequence of tasks with inhibitory neurons, it is beneficial to take into account the disparities in the degree to which updates to different parameters affect the layer's output distribution (Cornford et al. 2020) and hence forgetting. Specifically, since w ie and W4 affect the output distribution to a higher degree than W we reduce the learning rate for these weights after the first task (see Appendix).
Table I shows that models with feedforward neurons adhering to Dale's principle perform on par with standard neurons and can also further mitigate forgetting in conjunction with Active Dendrites when the quality of context signal is high (in case of Permuted-MNISTT, Note that this gain comes with considerably fewer parameters and context-dependent processing, we keep the number of neurons in each layer the same, and only excitatory neurons (~90%) are augmented with dendritic segments. For 20 tasks, Active Dendrite with Dale's principle reduces the parameters from 700 to less than ~64M parameters. We hypothesize that having separate populations within a layer enables them to play a specialized role. In particular, inhibitory neurons can selectively inhibit certain excitatory neurons based on stimulus, which can further facilitate the formation of task-specific subnetworks and complement the context-dependent processing of information by dendritic segments.
5

---

Published in Transactions on Machine Learning Research (04/2023)
Table 2: Effect of different levels of sparsity in activations on the performance of the model. Columns show the ratio of active neurons (I in k-WTA activation), and rows provide the number of tasks.
k
Rot-MNIST
Perm-MNIST
# #asks
0.05
0.10
0.20
0.50
0.05
0.10
0.20
0.50
5
92. 27
92.26=0.1
92.79=0.44
92.26=0.65
95.77=0.3s
96.32
90.29
74.51
10
70.78
71.95+1.64
73.32=0.69
71.61
95.06=0.29
93.45
72.68=12.55
41.33++..2
20
48.79=0.2
47.96+1.84
48.65sa.91
47.7 +0.91
92.40
84.28
+3.84
20.80
w/o Hetrogeneous Dropout
w/ Hetrogeneous Dropout
Task
Activation Count (x103)
Task
Task 3
2
Task 4
Task
7
9
11
13
15
17
19
21
23
25
1
3
11
13
15
17
19
21
23
25
Figure 2:
Total activation counts for the lest set of each (ask (y-axis) for a random set of 25 units in the second hidden layer of the model. Heterogeneous dropout reduces the overlap in activations and facilitates the formation of task-specific subnetworks.
4.2 Sparse Activations Facilitate the Formation of Subnetworks
Neocortical circuits are characterized by high levels of sparsity in neural activations (Barth Poulet 2012:
Graham Field 2006). There further evidence suggesting that neuronal coding of natural sensory stimuli should be sparse (Barth Poulet 2012: Tolhurst et al. 2009). This is in stark contrast to the dense and highly entangled connectivity in standard ANNs. Particularly for CL, sparsity provides several advantages: sparse non-overlapping representations can reduce interference between tasks (Abbasi et al. 2022 [yer et al. 2022 Aljundi et al. 2019), can lead to the natural emergence of task-specific modules (Hadsell et al. 2020
We study the effect of different levels of activation sparsity by varying the ratio of active neurons w-winner- take-all (k- (TA) activations (Ahmad & Scheinkman, 2019). Each hidden layer our model has a constant sparsity in its connoctions (randomly 50% of weights are set 0 at initialization) and propagates only the top-k activations (in Figure 1 KTA layer). Table shows that sparsity plays a critical role in enabling CL in DNNs. Sparsity in activations effectively reduces interference by reducing the overlap in representations. Interestingly, the stark difference in the effect of different levels of sparse activations on Rot-MNIST and Perm-MNIST highlights the importance of considering task similarity in the design of CL methods. As the tasks in Perm-MNIST are independent of each other, having fewer active neurons (5%) enables the network to learn non-overlapping representations for each task, while the high task similarity in Rot-MNIST can benefit from overlapping representations, which allows for the reusability of features across the tasks. The number of tasks the agent has to learn also has effect the optimal sparsity level. Tn Appendix, we show that having different levels of sparsity in different layers earlier layers learn general features, having higher ratio of active neurons can enable higher reusability and forward transfer. For the later layers, smaller ratio of active neurons can reduce interference between task-specific features.
further improve performance. As the
4.3 Heterogeneous Dropout for Non-overlapping Activations and Subnetworks
Information in the brain is encoded by the strong activation of relatively small set of neurons that form sparse coding. A different subset. of neurons is utilized to represent different. types of stimuli Graham

---

Published in Transactions on Machine Learning Research (04/2023)
Table 3: Effect of Heterogeneous dropout with increasing p values on different datasets with a varying number of tasks.
w/o
Dropout parameter (p)
Dataset
Tasks
Dropout
0.1
0.3
0.5
0.7
1.0
5
92.28
91.79a0.53
92.53
92.74
93.19so.az
93.42
Rot-MNNST
10
70.78
71.53
72.38
73.63=1.00
74.20
75.50=0.14
20
48.79 +0.07
48.57
48.91 10.05
49.84+0.t9
51.03 +0.31
51.11+000
5
95.77 00:33
95.70:00.29
95.97 10.44
96.40=a.2s
96.58 1o.17
96.48
Perm-MNIST
10
95.06 20
95. 95.23=th 23.22....0
95.65+0.20
95.54=0.20
95.74=0.22
95.94
20
92.40
92.83
93.20
92.82=a.0e
93.09
91.77=0.30
& Field 2006). Furthermore, there is evidence of non-overlapping representations in the brain. To mimic this, we employ Heterogeneous dropout (Abbasi et al. 2022 which in conjunction with context-dependent processing of information, can effectively reduce the overlap of representations, leading to less interference between tasks and, thereby, less forgetting. During training, we track the frequency of activations for each neuron in layer for a given task, and in the subsequent tasks, the probability of a neuron being dropped is inversely proportional to its activation counts. This encourages the model to learn the new task using neurons that have been less active for previous tasks. Figure D shows that neurons that have been more active (darker shade) are more likely to be dropped before k- WTA is applied. Specifically, let [ai], denote the activation counter of the neuron j in the layer after learning t tasks. For the learning task 1, the probability that this neuron is retained is given by:
1h+1 exp( max, [ai]] P)
where o controls the strength of enforcement of non-overlapping representations, with larger values leading to less overlap. This provides us with an efficient mechanism for controlling the degree of overlap between the representations of different tasks and, hence, the degree of forward transfer and interference based on the task similarities.
Table 3 shows that employing Hetcrogencous dropout can further improve the performance of the model. We also analyze the effect of the P parameter on the activation counts and the overlap in the representations. Figure 2 shows that Heterogeneous dropout facilitate the formation of task-specific subnetworks and Figure 3 shows the symmetric KL-divergence between the distribution of activation counts on the test set of Task and Task 2 on the model trained with different p values on Perm-MNIST with two tasks. As we increase the P parameter, the activations in each layer become increasingly dissimilar. Heterogeneous dropout provides a simple mechanism for balancing the reusability and interference of features depending on the similarity of tasks.
4.4 Layerwise Heterogeneous Dropout and Task Similarity
For an effective CL agent, it is important to maintain a balance between forward transfer and interference across tasks. As the earlier layers learn general features, a higher portion of the features can be reused to learn the new task, which can facilitate forward transfer, whereas the later layers Iearn more task-spocific features, which can cause interference. Heterogeneous dropout provides us with an efficient mechanism for controlling the degree of overlap between the activations, and hence the features of each layer. Here, we investigate whether having different levels of sparsity (controlled with the parameter) in different layers can further improve performance. As the earlier layers learn general features, having higher overlap (smaller p) between the set of active neurons can enable higher reusability and forward transfer. For the later layers, a lesser overlap between the activations (higher p) can reduce interference between task-specific features.
To study the effect of Heterogencous dropout in relation task similarity, we vary the incremental rotation, O Oncy in each subsequent task for Rot-MNIST setting with 5 tasks. The rotation of task is given by (r
7

---

Published in Transactions on Machine Learning Research (04/2023)
Overlap in Representations
11.0
W/o Hebbian Update
w/ Hebbian Update
Symmetric KL-Div (1e-3)
10.5
0.71
1:38
0:38
10.0
Context Vector
Task 2
0.79
1:08
-0.55
0.24
2.15
0.79
1.13
-0.56
0.24
4.99
9.5
Task 3
0.91
0.37
2.344
0.70
0.32
00.91
0.42
5..75
0.70
0.47
9.0
Task
0.66
1.64
0.72
8.0
0.0
0.1
0.3
0.5
0.7
1.0
5.0
Dendrite Segments
Dendrite Segments
Heterogeneous Dropout (o)
Figure 4: Dendritic segment activations of unit in layer for the context vectors of each task for a model trained on Perm- MNIST with 5 tasks. Hebbian update strengthens the association between the context and the dendritic segments, increasing the magnitude of the modulating signal.
Figure Effect of dropout on the overlap between the distributions of layer two aclivation counts for each task in WNIMT with 2 tasks. Higher p reduces the overlap.
Table 4: Effect of layer-wise dropout on Rot-MNIST with 5 tasks with varying degrees of incremental rotation (0mnc) in each subsequent task. Row 0 shows (ph, plz) the values for the first and second hidden layers, respectively.
Task Similarity (0inc)
p/,, p²2
2
16
32
0.1, 0.1
97.60
06.742a.u
91.79
74.99=1.1e
63.33
57.39=2.8
0.1, 0.5
97.77
97.02=o.11
92.39 20.39
75.56=1.46
18.17
57.05=2.1.
0.5, 0.5
97. 17.88
97. 97.22
92. 20.33
76.73=1.01
64. 10 18
58.35
0.5, 1.0
97.88+0.04
97.25 =0.11
92.87 40.39
76.87 =0.39
64.84 +0.65
59.40
1.0, 1.0
97.89 20.00
97.1920.00
93.4220.25
77.48=0.94
66.33
61.33 61.35=1190
1.0, 2.0
97.6840..
97.00
93.46.0.74
79.07 =0.07
68. 68.20
63.08
2.0, 2.0
97.42.0.17
97.002a.11
93.53
80.03
69.99
19.4..97
65. 74
2.0, 5.0
97.39
96.54
92.95
80.55
73.74
69.46
5.0, 5.0
96,8640.11
96. 12.1220..m
92.33
79.53=2.42
72.47+1.23
70.77=2.1
w/o Dropout
97.72+0.09
96.93
92.31 40.55
75.67=1.40
63.68+1.30
56.49
1)0ine. Table shows the performance of the model for different layerwise p values. Generally, heterogeneous dropout consistently improves the performance of the model, especially when the task similarity is low. For Oine 32, it provides ~55% improvement. As task similarity decreases (Oine increases), higher values of are more effective. Furthermore, we see that having different p values for each layer can provide additional gains performance.
4.5 Hebbian Learning Strengthens Context Gating
For biologically plausible ANN, it is important to incorporate not only the design elements of biological neurons but also the learning mechanisms it employs. Lifetime plasticity in the brain generally follows the Hebbian principle: neuron that consistently contributes to the firing of another neuron will build a stronger connection to that neuron (Hebb) 2005).
Therefore, we follow the approach Flesch al. 2023 complement eroor-based learning with the Hebbian update to strengthen the connections between contextual information and dendritic segments (Figure (bb).

---

Published in Transactions on Machine Learning Research (04/2023)
Each supervised parameter update with backpropagation is followed by Hebbian update step on the dendritic segments to strengthen the connections between the context input and the corresponding dendritic segment, which is activated. To constrain the parameters, we use Oja's rule, which adds weight decay to Hebbian learning (Oja 1982
Ux - Ur + -udd))
(6)
where Ti, is the learning rate, x is the index of the winning dendrite with weight "n and the modulating signal d = - for the context signal c. Figure 4 shows that the Hebbian update step increases the magnitude of the modulating signal from the dendrites on the feedforward activity, which can further strengthen context- dependent gating and facilitate the formation of tpsk-sp subnctworks. Table shows that this can consequently lead to improvement in results. Though the gains are not considerable in these settings, Table b shows the gains with Hebbian Learning are more pronounced in challenging CL settings with higher task and dataset complexity.
4.6 Synaptic Consolidation Further Mitigates Forgetting
In addition to their integrative properties, dendrites also play key role in retaining information and pro- viding protection against erasure (Cichon & Gan 2015 Yang et al. 2009). New spines that are formed on different sets of dendritic branches in response to learning different tasks protected from being climinated
through mediation of synaplic plasticily and structural changes that persist when learning new task (Yang
et al. 2009).
We employ synaptic consolidation by incorporating Synaptic Intelligence ZZenke et al. 2017) (details in Appendix) which maintains an importance estimate of each synapse an online manner during training and subsequently reduces the plasticity of synapses which are considered important for learned tasks. In particular, we adjust the importance estimate to account for the disparity in the degree to which updates to different parameters affect the output of the layer due to the inhibitory interneuron architecture in the DANN layers (Cornford et al. 2020). The importance estimates of the excitatory connections to the inhibitory units and the intra-layer inhibitory connections are upscaled to further penalize changes to these weights. Table [ shows that employing Synaptic Intelligence (+SC) in this manner further mitigates forgetting. Particularly for Rot-MNIST with 20 tasks, it provides a considerable performance improvement.
4.7 Experience Replay is Essential for Enabling CL in Challenging Scenarios
Replay of past neural activation patterns in the brain is considered to play a critical role in memory formation, consolidation, and retricval [Wlkerr & Stickgold 2004
McClelland et al. 1995.. The replay mechanism in
the hippocampus (Kumaran el. al. 2016) has inspired a series of rehearsa Sacaarra--bbsee -based approaches (Li & Hoiem
2017 Chaudhry et al.] 2019; Lopez-Paz & Ranzato 2017 Arani et al. 2022 that have been proven to be
effective challenging continual learning scenarios (Farquhar & Gal 2018 8 Hadsell et al. 2020). Therefore, to replay samples from previous lasks, we utilize small episodic memory buller that is maintained through Reservoir sampling (Vitter 1985). It attempts to approximately match the distribution of the incoming stream by assigning equal probabilities to each new sample to be represented in the buffer. During training, samples from the current task, (xs. y6) ~ Dr, are interleaved with memory buffer samples, (Im. vm) ~ M to approximate the joint distribution of tasks seen so far. Furthermore, to mimic the replay of the activation patterns that accompanied the learning event in the brain, we also save the output logits, ama across the training trajectory and enforce consistency loss when replaying samples from episodic memory. Concretely, the loss is given by:
C= (f(=:))) 9b) + aCots (f(m) (), ym) + ((f () )m
(7)
where f(:0) is the model parameterized by 0, Las is the standard cross-entropy loss, and a and B controls the strength of interleaved training and the consistency constraint, respectively.
Table [ shows that experience replay (+ER) complements context-dependent information processing and enables the model to learn the joint distribution well in varying challenging settings. In particular, the failure of the model to avoid forgetting in the Class-II. setting (Seq-MNIST) without experience replay
9

---

Published in Transactions on Machine Learning Research (04/2023)
suggests that context-dependent processing of information alone does not suffice, and experience replay might be essential. Adding consistency regularization (+CR) further improves performance as the model receives additional relational information about the structural similarity of classes, which facilitates the consolidation of information.
Table 5: Effect of each component of the biologically plausible framework on different Domain-IT. and Class- IL settings. For all experiments, we use a memory budget of 500 samples. We provide the average task performance and 1 sld of 5 runs.
Domain-IL
Class-IL
Method
Rot-FMMNIST
Seq-FMNIST
Seq-MNIST
GeI-ARI
Joint
98. 18.15
04.33ao.a
94.53
36.10 16..0
SGD
51.89
10.83
19.83
14.86x1.06
ActiveDANN
49.42.+085
21.46#1..5
19.97
16. 12.0.1
ActiveDANN + ER
80.99
77. 27.56
86. 26.88
28. 74.0422
Hebbian Update
82.16
78.02
88. 28.39
30. 10.122a.44
+ SC
82.5
78. 78.05.ao.m +0.00
88. 18.79+o.a
30.45
+ HD
83.97
78, 20..88
89.60 29.60
30. 30.2920.55
Bio-ANN
89.22 20.21
79.28
89.90
30.95
4.8 Combining the individual components
Having shown the individual effect of each of the brain-inspired components in the biologically plausible framework, here we look at their combined effect. The resultant model is referred to as Bio-ANN. Table [ shows that the different components complement each other and consistently improve the performance of the model. Our empirical results suggest. that employing multiple complementary components and learning mechanisms, similar the brain, may be an effective approach enable continual learning in ANNs.
4.9 Additional Results on Challenging CL settings
To further evaluate the versatility of the biological components on more challenging settings, we conducted experiments on Fashion-MNIST and grayscale version of CIFAR10. We considered both the Class-IL and Domain-II. settings. Seq-FMNIST, Seq-MNIST, and GeI-AR0 divide the classification into 5 tasks with 2 classes each, while Rot-FMNIST involves 20 tasks with each task involving classifying the 10 classes in each task with the samples rotated by increments of 8 degrees.
For brevity, we refer to Active Dendrites + Dale's principle as ActiveDANN. To show the effect of different components better (ActiveDANN without ER fails in the class-IL sctting), we consider ActiveDann + ER as the baseline upon which we add the other components. Empirical results Table show that the findings on MNIST settings also translate to more challenging datasets and each component leads to performance improvement. In particular, we observe that for more complex datasets, Hebbian learning provides sig- nificant performance improvement. The preliminary results suggest that the effect of biological mechanisms and architecture might be more pronounced more complex datasets and CL. settings.
5 Discussion
Continual learning is a hallmark of intelligence, and the human brain constitutes the most efficient learning agent capable of CL. Therefore, incorporating the different components and mechanisms employed in the brain and studying their interactions can provide valuable insights for the design of ANNs suitable for CL. While there are several studies that are inspired by the brain, they focus primarily on one aspect. Since the brain employs all these different components tandem, it stands to reason that their interactions, or complementary nature, are what enables effective learning instead of one component alone. Furthermore, the underlying framework within which these components are employed and the learning mechanisms might
10

---

Published in Transactions on Machine Learning Research (04/2023)
also be critical. The effort to close the gap between current AI and human intelligence could benefit from our enhanced understanding of the brain and incorporating similar mechanisms in ANNs. This is the fundamental question we aimed to study and bring to the attention of the research community at large.
We conducted a study on the effect of different brain-inspired mechanisms under a biologically plausible framework in the CT. setting. The underlying model incorporates several key components of the design principles and learning mechanisms in the brain: each layer constitutes separate populations of exclusively excitatory and inhibitory units, which adheres to Dalc's principle, and the excitatory pyramidal neurons are augmented with dendritic segments for context-dependent processing of information. We first showed that equipped with the integrative properties of dendrites, the feedforward network adhering to Dale's principle perform on par with standard ANNs, and provide considerable performance gains in cases where the the quality of context signal in high. Then we studied the individual role of different components. We showed that controlling the sparsity in activations using k-WTA activations and Heterogeneous dropout mechanism that encourages the model to use different set of neurons for cach task is an effective approach for maintaining a balance between reusability of features and interference, which is critical for enabling CL. We further showed that complementing error-based learning with the "fire together, wire together" learning paradigm can further strengthen the association between the context signal and dendritic segments that process them and facilitate context-dependen gating. To further mitigate forgetting, we incorporated synaptic consolidation in conjunction with experience replay and showed their effectiveness in challenging CL settings. Finally, the combined effect of these components suggests that similar to the brain, employing multiple complementary mechanisms in a biologically plausible architecture an effective approach to enable CL in ANN. It also provides framework for further study of the role of inhibition in mitigating catastrophic forgetting.
However, there are several limitations and potential avenues for future research. In particular, as dendritic segments provide an effective mechanism for studying the effect of encoding different information in the context signal, they provide an interesting research avenue as to what information is useful for the sequential learning of lasks and the effect of different context signals. Neuroscience studies suggest that multiple brain regions are involved in processing a stimulus and, while there is evidence that active dendritic segments receive contextual information that is different from the input received by the proximal segments, it is unclear what information is encoded in the contextual information and how it is extracted. Here, we used the context signal as in [Tyer et al. 2022), which aims to encode the identity of the task by taking the average input image of all the samples in the task. Although this approach empirically works well in the Perm-MNIST setting, it is important to consider its utility and limitations under different CL settings. Given the specific design of Perm-MNIST, binary-centered digits, and the independent nature of the permulations in each task, the average input image can provide a good approximation of the applied permutation, and hence efficiently encode the task identity. However, this is not straightforward for Rot-MNIST where the task similarities are higher and even more challenging for natural images, where averaging the input image does not provide a meaningful signal. More importantly, it does not seem biologically plausible to encode task information alone as the context signal and ignore the similarity of classes occurring in different tasks. For instance, it seemss more reasonable to process slight rotations of the same digits similarly (as in Rot-MNIST) rather than processing them through different subnetworks. This argument is supported by the performance degradation on the MoNTS sctting with active dendrites over standard ANNs. Idcally, we would like the context signal for different rotations of a digit to be highly similar. It is, however, quite challenging to design context signals that can capture wide range of complexities in the sequential learning of tasks. Furthermore, instead of hand engineering the context signal to bias learning towards certain types of tasks, an effective approach for learning the context signal in an end-to-end training framework is an interesting direction for future search. Another limitation of our framework would be the use of backpropagation which is widely criticized for being biologically implausible as it requires symmetric weight matrices in the feedforward and feedback pathways Xiao et al. and requires each neuron to have access to and contribute towards optimizing a single global objective function. The synapses in the brain, on the other hand, are unidirectional, and feedforward and feedback connections are physically distinct. The brain is also believed to perform localized learning. An interesting avenue for future research could be to extend our framework with more biologically plausible learning rules and study how the learning rule affects the interactions between the biologically plausible
11

---

Published in Transactions on Machine Learning Research (04/2023)
mechanisms. Future studies can also study the role of more plausible organization of neurons instead of fully connected, for instance, incorporation of population coding.
In general, our study presents a compelling case for incorporating the design principles and learning machin- ery of the brain into ANNs and provides credence to the argument that distilling the details of the learning machinery of the brain can bring us closer to human intelligence (Hassabis et al.) 2017 Hayes et al. 2021] Furthermore, deep learning is increasingly being used in neuroscience research model and analyze brain data (Richards et al. 2019). The utility of the model for such research depends two critical aspects: the
performance of the model and how close the architecture is to the brain (Cornford et al., 2020 Schrimpf ct al. 2020). The biologically plausible framework in our study incorporates several design components and
learning mechanisms of the brain and performs well in a (continual learning) task that is closer to human learning. Therefore, we believe that this work may also be useful for the neuroscience community in evalu- ating and guiding computational neuroscience. Studying the properties of ANNs with higher similarity the brain may provide insight into the mechanisms of brain functions. We believe that the fields of artificial intelligence and neuroscience intricately intertwined, and progress in one can drive the other as well.
References
Ali Abbasi, Parsa Nooralinejad, Vladimir Braverman, Hamed Pirsiavash, and Soheil Kolouri. Sparsity and heterogeneous dropout for continual learning in the null space of neural activations. In Conference Lifelong Learning Agents, pp. 617 628. PMLR, 2022.
Subutai Ahmad and Luiz Scheinkman. How can we be so dense? the benefits of using highly sparse representations. arXiv preprint arXiv: 11257, 2019.
Rahaf Aljundi, Marcus Rohrbach, and Tinne Tuytelaars. Selfless sequential learning. In International Conference on Learning Representations, 2019. URL
Flahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general continual learning method based on complementary learning system. In International Conference on Learning Representa- tions, 2022. URL https:://penreevie: wwwp:/ntt.ooeiiieeemee wix.uxxrrurrrr//...
Helen C Barron, Tim P Vogels, Timothy E Behrens, and Mani Ramaswami. Inhibitory engrams in perception and memory. Proceedings of the National Academy of Sciences, 1 114((6):6666-667) 2017.
Alison L Barth and James FA Poulet. Experimental evidence for sparse firing in the neocortex. Trends in
neurosciences, 35)533345,355 2012.
John M Bekkers Pyramidal neurons. Current biology, (242) 2011.
Marcus K Benna and Stefano Fusi. Computational principles of synaptic memory consolidation. Nature neuroscience, 19(12):1697-1706, 2016.
Martin Boerlin, Christian K Machens, and Sophie Denève. Predictive coding of dynamical variables in balanced spiking networks. PLoS computational biology, 9(11):e1003255 2013.
Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European conference on computer vision (ECCV), pp. 532 547, 2018
Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong
learning with a-GEM. In International Conference on Learning Representations, 2019. URL https:
//operreview wttp:///ww....tooommddiddddiaeaa sC5FFX
Joseph Cichon and Wen-Biao Gan. Branch-specific dendritic ca 2+ spikes cause persistent synaptic plasticity. Nature, 520(75446):80--155, 2015.
Jonathan Cornford, Damjan Kalajdzievski, Marco Leite, Amélic Lamarquette, Dimitri Michael Kullmann, and Blake Aaron Richards. Learning to live with dale's principle: Anns with separate excitatory and inhibitory units. In International Conference Learning Representations, 2020.
12

---

Published in Transactions on Machine Learning Research (04/2023)
Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning. arXiv preprint arXiv:185.00973, 2018.
Timo Flesch, David G Nagy, Andrew Saxe, and Christopher Sunmerfield. Modelling continual learning in humans with hebbian context gating and exponentially decaying task signals. PL.OS Computational Biology, = 2023.
Peter Foldiak. Sparse coding in the primate cortex. The handbook of brain theory and neural networks, 2003.
Kumihiko Fukushima. A self-organizing neural network model for mechanism of pattern recognition unaf- fected by shift position. Biol. Cybern., 36:193-2000 202, 1980.
Joaquin Fuster. The prefrontal corter. Academic press, 2015.
Daniel J Graham and David Field. Sparse coding the neocortex. Evolution of nervous systems, 181 187, 2006.
Raia Hadsell, Dushyant Rao, Andrei Rusu, and Razvan Pascanu. Embracing change: Continual learning in deep neural networks. Trends in cognitive sciences, 24(12):1028 1040, 2020.
Jun Han and Claudio Moraga. The influence of the sigmoid function parameters on the speed of backprop- agation learning. In International workshop on artificial neural networks, pp. 195 201. Springer, 1995.
Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick. Neuroscience- inspired artificial intelligence. Neuron, 95(2):245 258, 2017.
Tyler Hayes, Giri P Krishnan, Maxim Bazhenov, Hava T Sicgelmann, Terrence J Sejnowski, and Christo- pher Kanan. Replay in deep learning: Current approaches and missing biological elements. Newral Computation, 33(1)):2908 2950, 2021.
Donald Olding Hebb. The organization of behavior: neuropsychological theory. Psychology Press, 2005.
Carl Holmgren, Tibor Harkany, Björn Svennenfors, and Yuri Zilberter. Pyramidal cell commmncattio within local networks in layer 2/3 of rat. neocortex. The Journal of physiology, (551(1)339-111 1533, 2003.
Kevin Lee Hunter, Lawrence Spracklen, and Subutai Ahmad. Two sparsities are better than Unlocking the performance benefits of sparse-sparse networks. arXiv preprint arXiv:21 12.19966, 2021.
Alessandro Ingrosso and LF Abbott. Training dynamically balanced excitatory-inhibitory networks. PloS one, 14222070 2019.
Abhiram Iyer, Karan Grewal, Akash Velu, Lucas Oliveira Souza, Jeremy Forest, and Subutai Ahmad. Avoid- ing catastrophe: Active dendrites enable multi-task lcarning dynamic environments. Frontiers neu- vorobotics, 16, 2022.
James Kirkpatrick, Razvan Pascamu, Noil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barunn Barwinska, al. Overcomnng rophic forgetting in neural networks. Proceedings of the notional academy of sciences, 114(13):35211335) -522232 2017.
Dhireesha Kudithipudi, Mario Aguilar-Simon, Jonathan Babb, Maxim Bazhenov, Douglas Blackiston, Josh Bongard, Andrew P Brna, Suraj Chakravarthi Raja, Nick Chency, Jeff Clune, et al. Biological underpin- nings for lifelong learning machines. Nature Machine Intelligence, 4(3):196 210, 2022.
Dharshan Kumaran, Demis Hassabis, and James L McClelland. What learning systems do intelligent agents need? complementary learning systems theory updated. Trends in cognilive sciences, 20(7):512 534, 2016.
Louis Lapique. Recherches quantitatives sur l'excitation electrique des nerfs traitec comme une polarization. Jowrnal of Physiooogy and Patholoogy, 9:620- 635, 1907.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935 2242295-99977 2017.
13

---

Published in Transactions on Machine Learning Research (04/2023)
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. In Advances Wt neural information processing systems, pp. 6467-6476, 2017.
Arun Ma Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting single network to multiple tasks by learning to mask weights. Proceedings of the European Conference on Computer Vision (ECCV),
pp. 67-82, 2018.
Valerio Mante, David Sussillo, Krishna V Shenoy, and William T Newsome. Context-dependent computation by recurrent dynamics in prefrontal cortex. nalure, 503(7474):78 84, 2013.
James I. McClelland, Bruce I, McNaughton, and Randall C O' O'Reilly. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 10213):419, 1995.
Earl K Miller and Jonathan D Cohen. An integrative theory of prefrontal cortex function. Annual review of neurosciincee, 24(1):167-25) 202, 2001.
Brendan K Murphy and Kenneth D Miller. Balanced amplification: a new mechanism of selective amplifi- cation of neural activity patterns. Neuron, 61(4):635-645 2009.
Erkki Oja. Simplified neuron model principal component analyzer. Journal of mathematical biology, 15 (3):267 273, 1982.
Blake A Richards, Timothy P Lillicrap, Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Chris- tensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, et al. A deep learning framework for neuroscience. Nature neuroscience, 22(11): 1711:)1777777)) 2019.
Martin Schrimpf, Jonas Kubilius, Ha Hong, Najib Majaj, Rishi Rajalingham, Elias B Issa, Kohitij Kar, Pouya Bashivan, Jonathan Prescott-Roy, Franziska Geiger, et al. Brain-score: Which artificial neural network for object recognition is most brain-like? BioRriv, pp. 407007, 2020.
Markus Siegel, Timothy J Buschman, and Earl K Miller. Cortical information flow during flexible sensori- motor decisions. Science, 348(6241):1352-13388 1355, 2015.
Piergiorgio Strata, Robin Harvey, Dalc's principle. Brain research bulletin, 50(5):349-35) 349-300, 1999.
David J Tolhurst, Darragh Smyth, and Ian D Thompson. The sparseness of neuronal responses in ferret primary visual cortex. Journil of Neuroscience, 29(8):2355 2370, 2009.
Robin Tremblay, Soohyun Lee, and Bernardo Rudy. Gabaergic interneurons in the neocortex: from cellular properties to circuits. Neuron, (1(2):260-292 2016.
Carl Van Vreeswijk and Haim Sompolinsky. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science, 274(52033 1996.
Jeffrey S Vitter. Random sampling with reservoir. ACM Transactions on Mathematical Software (TOMS), 11(1):37-57, 1985.
Matthew Walker and Robert Stickgold. Sleep-dependent learning and memory consolidation. Newron, (1):121-133, 2004.
Will Xiao, Honglin Chen, Qianli Liao, and Tomaso Poggio. Biologically-phausin learning algorithms scale to large datasets. In International Conference Learning Representations.
Guang Yang, Feng Pan, and Wen-Biao Gan. Stably maintained dendritic spines are associated with lifelong memories. Nature, 962 2009.
Guang Yang, Cora Sau Wan Lai, Joseph Cichon, Lei Ma, Wei Li, and Wen-Biao Gan. Sleep promotes branch-specific formation of dendritic spines after learning. Science, 344(6188):1173 1178, 2014.
14

---

Published in Transactions on Machine Learning Research (04/2023)
Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continual learning of context-dependent processing in neural networks. Nature Machine Intelligence, 1(8):364-311 -722,, 2019.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. International Conference Machine Learning, pp. 3987-3995. PMTR, 2017.
15

---

Published in Transactions on Machine Learning Research (04/2023)
A Appendix
A.1 Related Work Biological Inspired Al
The human brain has long been source of inspiration for ANNs design (Hassabis et al. 2017 Kudithipudi et al, 2022). However, we have failed to take full advantage of our enhanced understanding of the brain, and there are fundamental differences between the design principles and learning mechanisms employed in the brain and ANNs. These differences may account for the huge gap in performance and behavior.
From an architectural design perspective, standard ANNs are predominantly based on the point neuron model [Lapique] 1907) which is an outdated and oversimplified model of biological computations that lacks sophisticated and context-dependent processing in the brain. Furthermore, neurons in standard ANNs lack adherence to Dale's s principle (Strata et al. 1999) to which most neurons in the brain adhere. Unlike the brain where presynaptic neurons have an exclusively excitatory or inhibitory impact on their postsynaptic partners, neurons in standard ANNs contain both positive and negative output weights, and signs can change while learning. These constitute two of the major fundamental differences the underlying design principle of ANNs and the brain.
Two recent studies attempt to address this gap. Cornford et al. (2000) incorporated Dale's principle into ANNs (DANNs) in a more biologically plausible manner and show that with certain initialization and regularization considcrations, DANNs can perform comparable to standard ANNs in object recognition tasks, which earlier allempls failed to do so. Our study exlends DANNs to the more challenging CL setting and shows that accounting for the discrepancy in the effect of weight changes in excitatory and inhibitory neurons can further reduce forgetting in CL. [yer et al.] (2022) propose an alternative to the point neuron model and provide an algorithmic abstraction of pyramidal neurons in the neocortex. Each neuron augmented with dendritic segments which receive an additional context signal and the output of the dendrite segment modulates the activity of the neuron, allowing context-dependent processing of information. Our study builds upon their work and provides a biologically plausible architecture characterized by both adherence to Dale's principle and the context-dependent processing of pyramidal neurons. This provides us with a framework to study the role of brain-inspired mechanisms and allows us to study the role of inhibition in the challenging continual learning setting, which is closer to human learning.
From a learning perspective, several approaches have been inspired by the brain, particularly for CL (uu- dithipudi et al. 2022). The replay mechanism in the hippocampus (Kumaran et al. 2016) has inspired series of behearsal-based approaches (Hayes et al. 2021: Hadsell et al. 2020) which have proven to be effective in challenging continual learning scenarios (Farquhar & Gal 2018). Another popular approach for continual learning, regularization--aase approaches Zenke et al. (2017); Kirkpatrick et al. (2017), has been inspired by neurobiological models that suggest that CL in the neocortex is based on a task-specific synaptic consolidation process that involves rendering a proportion of synapses less plastic and, therefore, stable over long timescales (Benna & Fusi, 2016 Yang et al., 2009]. While both approaches are inspired by the brain, rescarchers have mostly discounted the fact that the brain employs both of them in conjunction to consolidate information rather than in isolation. Therefore, the research in both of these methods has been orthogonal. Furthermore, they have been applied on top of standard ANNs which are not represen- tative of the complexities of the neuron in the brain. Our study employs replay and synaptic consolidation together in a more biologically plausible architecture and shows that they complement each other to improve performance.
Furthermore, our framework employs several techniques to mimic the characteristics of activations in the brain. As pyramidal neurons in the neocortex have highly sparse connectivity to each other (Hunter et all 2021 Holmgren et al. 2003) and only small percentage (<2%) of neurons are active for given stimuli neurons (Barth & Poulet, 2012), we apply k- k-winlertakko-ll (k- WTA) activations (Ahmad & Scheinkman 2019h to mimic activation sparsity. Several studies have shown the benefits of sparsity in CL (Abbasi el al.
2022 Mallya et al. 2018 Aljundi et al. 2019), they do not consider that the brain not only utilizes sparsity, it does so in an efficient manner to encode information. Information the brain is encoded by the strong activation of a relatively small set of neurons, forming sparse coding. A different subset of neurons is utilized to represent different types of stimuli (Foldiak 2003) and semantically similar stimuli activate similar set of
16

---

Published in Transactions on Machine Learning Research (04/2023)
neurons. Heterogencous dropout (Abbasi et al. 2022) coupled with k- &-WA activations aims to mimic these characteristics by encouraging the new task to utilize a new set of neurons for learning. Finally, we argue that it is important not only to incorporate the design elements of biological neurons but also the learning mechanisms that they employ. Lifetime plasticity in the brain generally follows the Hebbian principle (Hebb, 2005). Therefore, we follow the approach Flesch et al. (2023) to complement error-based learning with Hebbian update to strengthen the connections between contextual information and dendritic segments and show that it strengthens context gating.
Our study provides biologically plausible framework with the underlying architecture with the context- dependent processing of information and adherence to Dalc's principle. Additionally, it employs the learning mechanisms (experience replay, synaptic consolidation, and Hebbian update) and characteristics (sparse non-overlapping activations and task-specific subnetworks) of the brain. To the best our knowledge, we are the first to provide comprehensive study of the integration of different brain-inspired mechanisms in a biologically plausible architecture in CL setting.
A.2 Training Complexity
Here we discuss the training complexity of the different components in our framework. While Dale Prin- ciple adds additional learning parameters, subtractive inhibition allows for efficient implementation as the effective weight can be represented as W W W W Also as the number of inhibitory neurons is substantially lower than excitatory neurons, the additional parameters are low. The computational cost associated with Active Dendrites is twofold: the cost of creating the context signal for the task and the sub- sequent computations for the modulating signal in dendritic segments. Both of these depend significantly on the specific approach for evaluating the context signal and the dimensions of the context vector. The implemented approach takes an average input image of all the training samples for a task and therefore requires an additional pass through the training samples and the dimension of the context. vector is equal to the dimensions of the input image. More efficient approaches for creating context vectors would consid- erably reduce the computational cost associated with active dendrites. Activation sparsity, on the hand, substantially reduces the required computations for both the forward and backward pass (e.g. only 5% of the units are active in the forward pass, and therefore only those units will be trained) especially if compiler and hardware support sparse computations. Helerogeneous Dropout does not add a significant computation head as it only involves tracking the activation counts of the model and applying masking operation during training. Synaptic Intelligence maintains an online estimate of parameter importance without requiring an additional forward pass. It does however keep checkpoint of the model's previous weights for penalizing changes in weights.
We would like to emphasize that the brain employs a set of complex mechanisms in an efficient manner. As we bring more attention to employing similar mechanisms in ANNs and their complementary interactions, research this promising direction would inevitably lead to more efficient and effective learning.
A.3 Biological Components on Standard ANN
To further evaluate the applicability of the biological mechanisms in our framework and the role of underlying biologically plausible architecture, we conduct an ablation study on top of standard ANN. Table 6 shows the effect of adding activation sparsity using k- &-A to standard ANNs upon which then heterogeneous dropout (HD), synaptic consolidation (SC), and Experience Replay (ER) are added before finally combining all com- ponents -.N..)) Results show that activation sparsity considcrably improves the performance on Perm-MNIST. However, it reduces the performance on Rot-MNIST which can be altributed to the low degree of active neurons or the lack of forward transfer as the task similarity is high. Adding the biologi- cal components provide consistent performance improvements similar to the original scttings. The results are lower than ActiveDann experiments which provide further credence to the arguments that biological mechanisms top of biologically plausible architecture can be more effective.
17

---

Published in Transactions on Machine Learning Research (04/2023)
Table 6: Effect of each component of the biologically plausible framework on top of standard ANN on different datascts with varying number of tasks. We provide the average task performance and 1 std of five runs.
Rot-MNIST
Perm-MNIST
Method
Seq-MNIST
5
10
20
5
10
20
Joint
98.27
98.25-o.06
98.17 98.17-oa1
97.64-0.1
97.61
97.59...os
94.53.a.50
Standard ANN
93.7920.55
74.1320.8
31.89=0.30
77.96
76.42
65.18
19. 19.8320004
Activation Sparsity
92.36=0.30
70.61
48 48.0422.32
93,14
87 07.01
72.22
852.022
HD
92.72=20.54
=2.97
49.54.0.00
93.80=o.s1
89.07 #9.82
74.38
24. 28
+ SC
93.77 as.
74.68
51.45a:.m
96.07 20.11
95.30
12.861n.43
20.19au.a
+ ER
94.81
87.84=0.37
76.09=0.51
95.52=000a
93.11
86.34
81.44
Standard- Bio- ANN
96.66+0.09
93.32
88.57 =0.50
96.72=0.0
96.27=0.06
95.35+0.10
89.67 +0.46
Table 7: Forward Transfer with different dropout strength (p) on Rot-MNIST with tasks with varying degrees of incremental rotation (fize) each subsequent task. We report the average and standard deviation of runs.
Task Similarity (fine)
p p
2
4
8
16
24
32
w/o Dropout
88. 20.90
88.1020.95
86.5220.47
82.35
73.94
60.37
0.1, 0.1
28.85
88.05
86.9920.7s
82.45.0.ss
73.30
60.03
0.5, 0.5
88.96 no.75
88.31 10.91
87. 1 to.s0
82.33
73.66
59.94
1.0, 1.0
88.92
88.10
$6.89
81.97 11.97
72.80=o.52
59.54+0..0
5.0, 5.0
88.38
87. $7.62+0.pp
85. 43.
78.76+o.90
67.75+1.16
53.65
A.4 Forward and Backward Transfer
To further understand the effect of modularity on the stability and plasticity of the model, we evaluate the degree of forward transfer and forgetting (Chaudhry et al.) (018)) in the model depending on the task similarity and the degree of modularity enforced through different strengths of heterogeneous dropout. For- unseen tasks w.r.t. random guessing, whereas forgetting measures the performance degradation in subsequent tasks. We evaluate forgetting as the average difference between the maximum performance of the model on a task (when it is learned) and the final accuracy at the end of training on all tasks. We vary the incremental rotation, Oine: in cach subsequent task for Rot-MNIST setting with 5 lasks to simulate different degrees of task similarity with higher Oine corresponding to more dissimilar tasks. The degree of modularity controlled with different strengths of heterogeneous dropout (p parameter) with higher p values corresponding to lower overlap in representations and hence higher modularity.
ward transfer assesses whether the model is capable of improving
Table 7 provides the forward transfer while Table&proviin the backward transfer. We observe that generally as the tasks become more dissimilar, forward transfer reduces and there is more forgetting. Additionally, increasing the modularity (higher P values) generally leads to less forward transfer while reducing forgetting. Heterogeneous dropout provides us with an efficient mechanism to control the plasticity and stability of the model and an optimal balance between the stability and plasticity can provide the best performance. Interestingly, we observe that in certain higher similarity cases, moderate level of modularity sometimes increases the forward transfer.
A.5 Experimental Setup
To study the role of the different components inspired by the brain in a biologically plausible NN for CL and to gauge their roles in the performance and characteristies of the model, conduct all our experiments
18

---

Published in Transactions on Machine Learning Research (04/2023)
Table 8: Degree of Forgetting with different dropout strength (p) on Rot-MNIST with tasks with varying degroes of incremental rotation (fiee) in each subsequent task. We report the average and standard deviation of runs.
Task Similarity (finc)
ph, pl2
2
4
8
16
24
32
w/o Dropout
0.47
1.30 25
7.10=a.0e
29.96=a.04
43.06
50.96+1
0.1, 0.1
0.38
1.19
6.50
26.87=1.6
42.0320.40
50.45=1.07
0.5, 0.5
0.13
0.81
6.12
25.93
41.63
46.78
1.0, 1.0
0.12+0.0
40.10
5.52=1.04
23.77
38.29
44.36+2.43
5.0, 5.0
0.61
1.4620.22
5.81
21.82=1.97
27.39
30.95
Table 9: The selected hyperparamneters for the experiments showing the individual effect each component The base learning rate for all the experiments is 0.3 and the individual components use the same learning rates for Wi and We as (+ Dale's principle). For + SC experiments, we use Aw.. =10 and dw, =00. For ER experiments, we use memory budget of 500 samples.
(Table [
Dataset
#Tasks
+ Dale's Principle
+ Hebbian Update
+ SC
+ ER
+ ER + CR
5
n.. =302, nw., 33-3
n4 14.330 10=30-100 10
a=1, 8=0
a=1, 3=0.50
Rot-MNIST
10
n,. -3e22 1/W 1/.,,= =3-3
0
a=1, 3-0
a=1, 3=0.50
20
nW. =3e-3, 1/., =3e4
T4 n1 Te= 10
>= 1.00
a=1, 3=0
a=1, 3=0.50
5
r. -3e-2, 1/W,, 1/w.:=3e-2
T4 39-399
A=0.10
a=1, 3-0
a=1, 3=0.50
Perm-MNIST
10
TW. =30-2, fW., =30-
n1 3c-30-06
A=
a=1, 8=0
a=1, 3=0.50
20
nW. =3 nW., 3
T42 3e-39-09
==0.10
a=1, 3=0
a= 3=0.50
Seq-MNIST
5
lW. -3e-2, 7/, 7/... 3e-3
x=
a=11 3=0
a=1, 3022
under uniform settings. Unless otherwise stated, we use a multi-layer pereeption (MLP) with two hidden layers with 2048 units and k-WTA activations. Each neuron is augmented with N dendritic segments where N corresponds to the number tasks and the dimensions correspond to the dimensions of the context vector that correspond to the input image size (784 for all MNIST-based settings). The model is trained using an SGD optimizer with 0.3 learning rate and a batch size of 128 for 3 epochs on each task. We set the weight sparsity to 0 and set the percentage of active neurons to 5%. For our experiments involving Dale's principle,
maintain the same number of total units in each layer divided into 1844 excitatory and 204 inhibitory units. Only the excitatory units are augmented with dendritic segments. Importantly, we use the initialization strategy and corrections for the SGD update as posited in Cornford et al. (2000) to account for the disparitics in the degree to which updates to different parameters affect the layer output distribution. Updates to inhibitory wnit parameters are scaled down relative to update of excitatory parameters. Concretely, the gradient updates to Wie were scaled by yne" and We by d'1 where ne are the number of excitatory neurons in the layer and d is the input dimension of the layer. Furthermore, to select the hyperparameters for different settings, use small validation set. Note that, as the goal was not to achieve the best possible accuracy, but rather to show the effect of each component, we did not conduct an extensive hyperparameter search. Table g provides the selected hyperparameters for the effoct of individual component experiments in Table | and Table 10 provides the selected hyperparamcter for ANo experiments. We report the mean accuracy over all lasks and 1 sid over three diflerent random seeds.
A.6 Effect of adjusting for the inhibitory weights
The inhibitory interneuron architecture of DANN layers introduces disparities in the degree to which updates to different parameters affect the layer's output distribution, e.g. if single element of Wie is updated, this
19

---

Published in Transactions on Machine Learning Research (04/2023)
Table 10: The selected hyperparamneters for Bio-ANN experiments in Table ! rate for each setting Dalc's principle (Table 91.
We use the same learning
Dataset
# #Tasks
A
p
B
30-8
0.25
0.1
1
0.5
Rot-MNIST
3e-8
0.1
0.3
0.5
20
3e-8
0.1
0.3
1
0.5
3e-6
0.1
0.1
1
0.5
Perm-MNIIS
10
3e-8
0.1
0.3
1
0.5
20
3e-8
0.1
0.3
1
0.5
Seq-MNIST
5
3e-6
0.1
0.1
1
0.25
Table 11: Effect of adjusting the learning rates of Wie and We the end of the first task on different datasets with varying number of tasks.
Rot-MNIST
Perm-MNIST
Viie
?Wei
5
10
20
5
10
20
3c-1
3e-1
92.12aa.3t
70.86
46.30
95.78
04.73
92.6720..a
3e-2
3e-2
92.
23.00.53
70.23
47.53=1.70
95. #0.33
95.06 20.06=.oo
91.63
3e-2
3e-3
92.28 1.0.7
70.78
47.3
95.68 +0.14
94.96 o...0
92.40 1o.3s
3e-3
3e-3
92.34
71.27
47.81=1.10
95.70
04.44
92.02+0.19
3e-3
3c-4
92.03
70.79
48.79
95.90
94. 94.1940.44
91.04.0.54
has effect each element of the layer's output. An inhibitory weight update of ö to Wie changes the model distribution approximately n, times more than an excitatory weight update of 5 to W.. (Cornford et al. 2020). The effect of these disparities would be even more pronounced in CL setting as large changes to the output distribution when learning a new task can cause more forgetting previous tasks. To account for these, we further reduce the learning rate of W,e and Wşi after learning the first task. Table shows that accounting for the higher effect of inhibitory neurons can further improve the performance of the model in majority of the settings. It would be interesting to explore better approaches to account for the aforementioned disparitics, which are tailored for CL and consider the effect forgetting.
A.7 Effect of adjusting for scaling the SI importance estimate for inhibitory weights
Similar to adjusting the learning rate of inhibitory weights, we check whether scaling up the importance estimate of inhibitory neurons can further improve the effectiveness of synaptic consolidation in reducing forgetting. Table 12 shows that scaling the importance estimate accordance with the degree to which inhibitory weights affect the output distribution, and hence forgetting, further improves the performance in majority of cases, especially for a lower number of tasks. This suggests that regularization methods designed specifically for networks with inhibitory neurons is a promising research direction.
A.8 Effect of Sparsity
To further study the effect of different levels of sparsity in activations and connections, vary the number of weights randomly set to zero at initialization (Sw E {0. 0.25, 0.50)) and the ratio of active neurons (K' 122)) in each hidden layer. Table 13 shows that sparsity in activation plays a critical role in enabling CL in ANNs. Interestingly, sparsity in connections play a considerable role Perm-MNIST with higher levels of active neurons (2 0.2). Furthermore, exploring finer differences in activation sparsity of different layers may further improve the performance. Similar to heterogencous dropout, we show the effect of activation sparsity in relation to task similarity in Table 14 Similar tasks (lower Oanc) benefit. from higher number of active
20

---

Published in Transactions on Machine Learning Research (04/2023)
Table 12: Effect of scaling the importance estimate for Wie and We to reduce the parameter shift in the inhibitory weights Rot-MNIST and Perm-MNIST datasets with varying number of tasks.
Rot-MNIST
Perm-MNIST
A
AW
Awer
5
10
20
20
1
1
92.92
75.53=1.46
12.16
26.59
26.20
95.69a0.u
0.1
10
10
92.77 40.27
74.70
4..04
96.57-0.3
96.18
95.64-o.15
100
=2.50
74.67=1.27
52.55
96.67 =0.23
96.26=o.20
95.61=0.10
1
92.77 #0.69
76.30
55.05
96.62
96.07=0.0z
95.36=0.24
0.25
10
10
93.54 54.44..0 00.70
75.44
54.93
96.54
96. 96.22+o.10
95.03
10
100
93. 23.40=z.. 20.8
75.87=1.a5
55.06+1.77
96.65
96.36=0.u
95,18=0.85
20.73
75.24
60.24+2..0
96.22
95.97 #0.06
92.94
0.5
10
10
93. 23.1320.24
75.95
59. 11.35
96. 06.2442..m
95.81 10..0
92.86=0.at
10
100
92.85 +0.42
75.85+1 +1.18
58.94
96.36 +o
96.02 +0.27
93.48
w/o SC
92.28 20.27
70.78 20.23
48.79.o.27
95.77 20.33
95.06
92.40
Table 13: Effect of different levels of sparsity in activations (ratio of active neurons, (k"1 k22) in 1st and 2nd hidden layer respectively) and connections (ratio of zero weights, Sw) on Rot-MNIST and Perm-MNNIST with increasing number of tasks. The best performance across the different sparsity levels for each task is in bold.
Activation Sparsity (k21,
# Tasks
Sw
(0.05, 0.05)
(0.1, 0.05)
(0.1, 0.1)
(0.2, 0.1)
(0.2, 0.2)
(0.5, 0.2)
(0.5, 0.5)
0.00
92.25 02.28.o..7 0.97
92.63 o0.40
92.26 0.
92.25
0.71
92.79 a..se
92.51 0.50
92.26 0.03
5
0.25
91. 1o.58
91.50
92.33
91.75
92.60....a
91.23 1.63
91.66
0.50
90. 10.78
91.15an.34
91.252a.14
91.25=a..z
90.67
90.75=a..0
90.41=1.4
Rot-MNIST
0.00
70.7820.28
71.162u.51
71.95
72.22
73.32
71.89
71.61=0.70
10
0.25
70. 20.76
=1.42=0.08
71.8420.72
73.22=1.84
72.58
22.23
71.23=0.84
0.50
69.61 10.50
70.59 10.62
70.94
72.05
0..4
72.25 1.40
71.37
70.85
0.00
48.75 L0.37
48.01
47.96
48.33
48.65
48.19
47.71
20
0.25
47. 17.722a.a 22.0.as
48.61:2m50
48.41
48.53=1.77
48.30=uat
18.2921.55
47.11=a.44
0.50
46. 20.0.
47.15
48.02
48.17=1.42
48.30
47. 17.66=1.55
47.73=0.51
0.00
95. 20.33
95.55
96.43
95.85=0.20
90.29 00.2922.07
88, 98,18
74.51=19.56
0.25
95. 25.45
95. 2.
95.65
95. 75.0.aa
93.1321.20
87.49
75.97=5.0
PermI-MNIST
0.50
93. 13.95
94.19
94.90...22
94.22 1.19
94.05=a.av
91.02
83.94
0.00
95.06 20.29
94.08 20.99
94.38
89.54m2227
78.91=s.2
76.44_7.99 T.99
35.86=2.s
10
0.25
94.51 10.12
93.52
03.6220.64
88,58=2.1
84.94
69.32
59.5329.s7
0.50
92. 22.622
90.31
92, 2920.7
88, 08.48=2..2
82.57=2.40
74.42=2.84
69.31=5.60
0.00
92. 12.400z..s
89.41=1.79
84.28
73.29=22
63.84 03.84=s.45
58.85
20.80
20
0.25
90. 00..55o.00
89.22=0.04
84. 14.29922.1
75. 75.82=6.ck
67. 07.26=1.22
63.96-1.19
41.60=2.66
0.50
37.31
84.90=1.90
83.72=0.65
69.62m5.e8
66.27=a.74
64.42=a.s
51.92=a.79
neurons that increase forward transfer, while dissimilar tasks (higher 0ine) perform better with higher activation sparsity that can reduce the overlap in representations.
21

---

Published in Transactions on Machine Learning Research (04/2023)
Table 14: Effect of different levels of activation sparsity on Rot-MNIST with 5 tasks with varying degrees of incremental rotation (0mee) in each subsequent task. Row 0 shows (Kt1, k22) the ratio of active neurons in the 1s and 2nd hidden layers, respectively.
Task Similarity (0inc)
2
4
8
16
24
32
0.05, 0.05
97.54.0.0s
96.562o.14
91.95
75.13
63.1440.22
57.01
0.1, 0.05
97.57
96.84
92.49
76.15+1.2
64.03
58.56=1.17
0.1, 0.1
97.81=2
96.84
92.28
76.80
64.91
58.62=1.70
0.2, 0.1
97.44
96.88
92.47
25.79
64.38
58.34+1.54
0.2, 0.2
97.88
97.27 7 10
92.79 20.50
76.22
64.30
57.22 57.22=1.00
0.5, 0.2
97.67
96.92 +....
92.61 +0..5
75.66+0.95
63.86+0.61
56. 1 1.220
0.5, 0.5
97.67
97.03
92.29
74.55
62.37
53. 13.3.22
22

---

Published in Transactions on Machine Learning Research (04/2023)
Algorithm Bio-ANN: A biologically plausible framework for CL.
Input: Data stream D; Learning rates 77, ?ie m:.;; Hebbian learning rate mhi Heterogeneous dropout
p; Synaptic consolidation weights X, Xw... Aw.s 7; Experience replay weights a, Initialize:
Model weights 8, Reference weights 0. = 0. Task prototypes C- = 0
Heterogeneous dropout: Overall activation counts A, = 0, Keep probabilities P. =
Memory buffer M + {
Synaptie Intelligence: w = 0,
Sample task from data stream
for D, E D2,,.,,. Dr} do
b Task context
Evaluate context vector (Eq. 2
Cr [D-] EzED, x
3: Update the set of prototypes:
C, + (C+,cr}
D. Train on task 7
4: while Training do Sample data: (It, yb) Dr and (Ims Um. 2m) M
Task specific loss
Get the model output and activation counts the current task batch:
2b, ab F(rs, Gr;,,P) # Apply Heterogeneous dropout
7:
Calculate task loss:
L. = (2, yb)
8:
Update overall activation counts:
A, + UpddateActivationCouus//
b Experience replay
Infer context for buffer samples (Eq.
Cm min ||æ' C, ll2
10:
Get model output on buffer samples:
2 = F(xm,
# Disable Heterogencous dropout
11:
Calculate replay loss:
Cer alolo ym) + B(z 2m)²
Synaptic regularization
12:
Calculate SI loss:
Cen = Dacti (0 0z)²
13:
Calculate overall loss and clip the gradient between 0 and 1:
L = C, + Ler + Leo
VoL = Cli 0,
Update Models
14:
SGD update: UpdateModel( VaL, !J, IW.. 1J.,) Hebbian update dendritic segments: U HobbianStep((c..com cm), U)
15:
16:
17:
Update small omega: w = w V822
Update SI parameter
18:
M + Reservoir(M, (Io, yb, =n))
b Update memory buffer (Algorithm 2
19:
Task Boundary
20:
Update keep Probabilities (Eq 5
P, expp -=
21:
Update SI Omega and reference weights and reset small omega:
Ω Ω + (,-,,1.
w
0. = 0
22:
Scale up importance for inhibitory weights aoo ScalcUpInhib(1,, Aw.. Aw.))
return
23

---

Published in Transactions on Machine Learning Research (04/2023)
Algorithm Reservoir Sampling
Input: Memory Buffer M, Buffer Size B, Number of examples so far N, data points y,2)
1: if B > N then
b Check if memory is full
2:
M[N] + (1)
3: else
Select a sample to replace
4: 12 = Samptelladommnnegrr(nnn-o,,, M)
if < B then
M[n] - (x, y, 2)
return M
24


Reviews:
[{"review_id": "yFyj7aVFXD", "reviewer": "Reviewer_baVv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\nThe authors study the effect of combining various biologically plausible features, which to my knowledge hasn't been\ndone before. It is also useful to see that various different biologically plausible features don't interfere with each\nother and prevent CL.\n\nThe authors also do extensive experiments, which provides a lot of information on which features are important,\nalthough the analysis is not fully thorough -- see below.\n\nThe candid discussion of biological background on the use of contextual information is very insightful and informative.\n\n### Weaknesses\n\nOverall, the conclusions of and the outcomes from this work are not clear. The authors try out a number of features, but\n   some of them don't clearly contribute to improved performance in the continual learning (CL) task and some of them do. Is the\n   goal to add all known biological mechanisms and say that CL is still possible? Or is the goal to isolate those features\n   that actually make a different in CL tasks? The focus of the paper requires sharpening.\n\n1. Baseline performance on a ANN without any of the features is not compared (e.g. Table 1).\n2. The study of all the features such as Hebbian Updates, synaptic consolidation etc. using a network with active\n   dendrites as the basis. But it would have been useful to see the effect of these features on CL performance even\n   without active dendrites to know if they absolutely require active dendrites.\n3. There are some potentially biologically implausible mechanisms which are not discussed in sufficient detail -- the use of back-propagation for instance.\n4. It's not clear if layer-wise heterogeneous dropout and Hebbian learning actually contribute to any improvement in CL\n   performance overall.\n5. For a study which focuses on studying the importance of various features/mechanisms for CL, reporting the task\n   performance over only 3 runs makes it really hard to judge how robustly these mechanisms contribute to the CL\n   performance.\n\nMinor:\n- Why are 90% inhibitory neurons used instead of the more widespread and presumably more biologically supported 80%?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors explore the effect of combining various biologically plausible architectural features on the ability of the network to continually learn. Specifically, they combine active dendrites, Dale's principle, k-WTA activations, heterogeneous dropout (both layer-wise and otherwise), Hebbian learning, Synaptic consolidation (using synaptic intelligence), experience replay with consistency regularization. They find that adding some of these features can lead to improvements in the continual learning ability of the network.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would request the following changes to make the manuscript stronger and more focused.\n1. Run more than 3 runs for each feature.\n2. Report baseline with ANN, add ablation studies of various features without active dendrites as well.\n3. Either an explanation should be given for using features that don't contribute to CL performance or they should be\n   discarded.\n4. The use of backprop should be more prominently mentioned and discussed.\n\nMinor:\n- Table 1: Please add explanations for the various acronyms or descriptions of what each column (5, 10, 20) denotes.\n- Fig. 1(a): arrows are missing on the green line, $W_{ie}$ and $W_{ei}$ seem to be referring to the same connection.\n- In section 4.3, not clear what the Table 3 results contribute over those already in (Abbasi et al. 2022). Please\n    explain or remove redundant results.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7FREhxvlnC", "reviewer": "Reviewer_PmJe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "+ Very interesting framework and the authors have shown the effectiveness in different CL MNIST benchmarks\n+ The bio-plausibility of the method points to some interesting learning behaviors that is not shown with previous CL methods.\n\nSome of my comments on the work are:\n- What is the complexity of training in the author's approach? A body of CL works today [1, 2] also try to look into how much memory compression and how much cost is being used to train the model in a CL manner. A discussion on that ll be helpful. \n[1[ Venkatesha Y, Kim Y, Park H, Li Y, Panda P. Addressing Client Drift in Federated Continual Learning with Adaptive Optimization. Available at SSRN 4188586. 2022 Mar 24.\n[2] Saha G, Garg I, Roy K. Gradient projection memory for continual learning. arXiv preprint arXiv:2103.09762. 2021 Mar 17.\n\n- How does the dendritic connections addition/removal work in terms of training complexity? If your network is changing, then does that mean you are reinitializing the model, or starting from a previously stored point?\n\n- Finally, I was wondering if the authors can comment on how will their approach scale with larger datasets or increasingly complex CL tasks.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors  consider a biologically plausible framework that constitutes separate populations of exclusively excitatory and inhibitory neurons that adhere to Dale’s principle, and the excitatory pyramidal neurons are augmented with dendritic-like structures for context-dependent processing of stimuli. They conduct a comprehensive study on the role and interactions of different mechanisms\ninspired by the brain, including sparse non-overlapping representations, Hebbian learning, synaptic consolidation, and replay of past activations that accompanied the learning event. Their study suggests that the employing of multiple complementary mechanisms in a biologically plausible architecture, similar to the brain, may be effective in enabling continual learning in ANNs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please provide a discussion on the weaknesses highlighted above.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "IHRj1PKqkJ", "reviewer": "Reviewer_kt2N", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "In general, although I think the paper is certainly valuable for the community and the experiments seem mostly sound, there are several drawbacks and limitations that need addressing.\n\n\nStrengths:\n- I think the topic of this study is interesting and useful for the community. As the authors note, many continual learning approaches are biologically inspired and the field has strong connections to neurobiology. However, there are few studies that thoroughly discuss these links and in particular the way different methods work when combined. As such, I think this paper should be of interest to TMLR's audience.\n- Authors consider many different methods of various types in their study (e.g. regularization-based Synaptic Intelligence, sparsity-enforcing k-WTA and rehearsal-based experience replay). I appreciate that since many papers only focus on a single family of methods in their investigations.\n- The paper includes many ablation studies showing how different values of hyperparameters impact the results, allowing the reader to build a better intuition of the problem and the performance of the tested approaches.\n- The paper is well written and nicely discusses related work. I appreciate the detailed discussion and the extensive related work section in the appendix. \n\nWeaknesses:\n- Many of the methods tested in this paper enforce modularity in some ways (e.g. sparsity-enforcing k-WTA). This of course limits forgetting as it reduces interference, but at the same time it raises the question of whether the network is able to reuse the knowledge between these modules. As such, I don't think the issue of transfer between different tasks has been analyzed well enough in this paper. In my opinion, the question of \"can we efficiently reuse prior knowledge\" should be treated in continual learning with equal importance as the question \"can we avoid forgetting?\", and the paper mostly focuses on the second one.\n- Investigating the performance of the combinations of methods is interesting, but at the same time, I'm not sure if the paper sufficiently covers the space of possible ways how to combine them. In particular, Table 1 only contains the base ActiveDANN model combined with additional methods. It would be interesting to see the synergies and incompatibilities of different approaches (e.g. what if we only use Synaptic Intelligence and Experience Replay). I raise this issue as checking combinations of different methods seems to be a major point of the paper.\n- Although the study mostly focuses on biologically-plausible approaches, I think it should also consider the standard baselines used in continual learning such as a standard MLP trained with SGD. Such an approach is only tested in Table 2 in the Appendix and then it is only compared to biologically plausible approaches which include ER, which I think is an unfair comparison. Given these points, I don't think the following statement in Discussion is true: \"We first showed that equipped with the integrative properties of dendrites, the feedforward network adhering to Dale’s principle not only performs as well as standard ANNs, but also provides gains.\"\n- The empirical evaluation is mostly conducted using the MNIST benchmark. In continual learning literature, and in deep learning in general, it is no longer considered an interesting benchmark by itself -- many methods that perform well on MNIST do not scale well to tasks better representative of the real world. The authors do include some Fashion-MNIST results in the appendix, but by itself, I still think this is not enough. I do understand that the presented method might not work well with convolutional networks so it might be difficult to show interesting results on e.g. CIFAR-10. Still, given the experimental protocols in continual learning, I have to say that the datasets used here are too simplistic.\n- Also, the authors mostly consider the Incremental Domain setting. There is a single experiment with the Incremental Class scenario in Table 1, but it seems that Experience Replay is the only approach that really works here. This is a limitation in the scope of the study.\n\nAdditional comment: The paper focuses on studying how to combine the strengths of previously proposed biologically-plausible continual learning methods, rather than proposing new solutions. This is not a weakness, especially given TMLR's guidelines, but it's worth noting.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper investigates how combinations of biologically plausible methods perform in continual learning. The authors use approaches proposed in previous works such as architectures based on Dale's principle, sparsity-enforcing activation functions (k-WTA), heterogeneous dropout, synaptic intelligence and experience replay. They test the listed methods on continual versions of the MNIST benchmark, focusing in particular on their combined performance. The paper also includes various ablation studies (e.g. impact of the task similarity on the correct choice of dropout temperature).", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would like to ask the authors to address the weaknesses listed in the section above. However, as I mentioned, I think the paper is already useful as it is, so I'd be in favor of accepting the paper if the authors explicitly listed the issues above as limitations without introducing the changes.\n\nSuggested improvements (corresponding to the list of weaknesses)\n- Report metrics such as forward transfer in the study. Additionally, include an additional experiment checking transfer depending on task similarity and degree of modularity in the model.\n- Include more combinations of the tested methods in the empirical study.\n- Include a standard ANN as a baseline. Ideally, show how it performs with some of the suggested improvements (k-WTA, experience replay, Synaptic Intelligence).\n- Extend results on Fashion-MNIST. Ideally, run some experiments on datasets such as CIFAR-10, even with linear (rather than convolutional) layers.\n- Include more results on Incremental Class tasks. Incremental Task would also be a nice addition, but it's usually regarded as easier than Incremental Domain so it's probably not necessary.", "broader_impact_concerns": "I don't think the paper needs a broader impact statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers all appreciated that this was a comprehensive comparison of the role and interactions between a diverse set of bio-inspired innovations to ANNs that may improve continual learning.  They believed this is of value for the community.

Based on the initial paper, all reviewers wanted to see how the various innovations fared independently (in the absence of active dendrites).  This was added in what is now Table 6 (in appendix).  
- I would recommend pointing to this table from the caption to Table 1.

Reviewers baVv and kt2N made a series of suggestions for changes and additional analyses, most of which, the authors have performed. 

Reviewer kt2N was mostly happy with the changes, but made a few final requests that were accidentally commented without the authors' visibility.  I am copying these requests here and would encourage the authors to make updates reflecting these. 
- "Could you explicitly write down the definitions of forward/backward transfer used in Tables 6 and 7? It's not 100% clear to me what you are measuring."
- "The result that active dendrites actually harm performance on Rotated-MNIST are quite surprising. This is not necessarily a result that disqualifies this work, but I think this result should be clearly reflected in the text. For example, in Section 4.1 the authors say: "Table 1 shows that models with feedforward neurons adhering to Dale’s principle perform as well as the standard neurons and can also further mitigate forgetting in some cases". This is not true for Rotated-MNIST. Similar corrections are needed in other sections."

There was a bit of discussion between authors and reviewers (especially reviewer baVv) about whether specific innovations are significant. From looking at table 1, which does not have indications of statistically significant improvements, I'm honestly not clear whether the Hebbian update is actually reliably improving performance by a significant amount.  In one place the authors added "Table 1 shows that this can lead to improvement in results."  However, since there are multiple comparisons, some of which go up and some go down, and with small effect sizes, it would be appropriate to perform significance testing, tone down the claims, and/or be clear that the effect size is quite small.

In addition to the points made by reviewers, I had a few minor comments about the presentation. A few terms are not introduced before their acronyms are used or before they are used in captions.
- "Heterogeneous dropout" is repeatedly mentioned in captions, but not fully introduced until the evaluation setting section.
- Table 1 would benefit from including all acronym definitions in the caption.  Many haven't been defined yet.

Decision: Accept (poster)