Multi-Synaptic Cooperation: A Bio-Inspired Framework for Robust and Scalable  Continual Learning
Authors: Penghui Li
Zhuang Ma
Yunliang Zang
Qiang Yu
Venue: ICLR 2026 Poster (2026)
Keywords: Continual Learning
Catastrophic Forgetting
Brain-inspired Computing
Multi-Synaptic Cooperation
Primary area: applications to neuroscience & cognitive science
TL;DR: We propose the Multi-Synaptic Cooperative Network (MSCN) for continual learning. MSCN achieves higher accuracy and superior robustness compared to state-of-the-art baselines.

Abstract: Continual learning aims to acquire new knowledge incrementally while retaining prior information, with catastrophic forgetting (CF) being a central challenge. Existing methods can mitigate CF to some extent but are constrained by limited capacity, which often requires dynamic expansion for long task sequences and makes performance sensitive to task order. Inspired by the richness and plasticity of synaptic connections in biological nervous systems, we propose the Multi-Synaptic Cooperative Network (MSCN), a generalized framework that models cooperative interactions among multiple synapses through multi-synaptic connections modulated by local synaptic activity. This design enhances model representational capacity and enables task-adaptive plasticity by means of multi-synaptic cooperation, providing a new avenue for expanding model capacity while improving robustness to task order. During learning, our MSCN dynamically activates task-relevant synapses while suppressing irrelevant ones, enabling targeted retrieval and minimizing interference. Extensive experiments across four benchmark datasets, involving both spiking and non-spiking neural networks, demonstrate that our method consistently outperforms state-of-the-art continual learning methods with significantly improved robustness to task-order variation. Furthermore, our analysis reveals an optimal trade-off between synaptic richness and learning efficiency, where excessive connectivity can impair circuit performance. These findings highlight the importance of the multi-synaptic cooperation mechanism for achieving efficient continual learning and provide new insights into biologically inspired, robust, and scalable continual learning.

Published as a conference paper at ICLR 2026
MULTI-SYNAPTHE COOPERATION: A BIO-INSPIRED FRAMEWORK FOR ROBUST AND SCALABLE CONTIN-
UAL LEARNING
Penghui Li:-;; Zhuang Ma Yunliang Zang"; Qiang Yut.3t
School of Artificial Intelligence, CCA Lab, Tianjin University, Tianjin, China
2 School of Future Technology, Tianjin University, Tianjin, China
College of Intelligence and Computing. Tianjin University, Tianjin, China
Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin, China
( [ ipennhuu, ma mazhanng, yunl punliangzang, yuqiang @t ju. edu. on
ABSTRACT
Continual learning aims to acquire new knowledge incrementally while retain- ing prior information, with catastrophic forgetting (CF) being a central challenge. Existing methods can mitigate CF to some extent but are constrained by limited capacity, which often requires dynamic expansion for long task sequences and makes performance sensitive to task order. Inspired by the richness and plasticity of synaptic connections in biological nervous systems, we propose the Multi- Synaptii Cooperation Network (MSCN), generalized framework that models cooperative interactions among multiple synapses through multi-synaptic connections mod- ulated by local synaptic activity. This design enhances model representational capacity and enables task-adaptive plasticity by means of multi-synaptic coop- eration, providing new avenue for expanding model capacity while improving robustness to task order. During learning, our MSCN dynamically activates task- relevant synapses while suppressing irrelevant ones, enabling targeted retrieval and minimizing interference. Extensive experiments across four benchmark datasets, involving both spiking and non-spiking neural networks, demonstrate that our method consistently outperforms state-of-the-ar continual learning methods with significantly improved robustness to task-order variation. Furthermore, our analy- sis reveals optimal trade-off between synaptic richness and learning efficiency, where excessive connectivity can impair circuit performance. These findings high- light the importance of the multi-synaptic cooperation mechanism for achieving efficient continual learning and provide new insights into biologically inspired, robust, and scalable continual learning.
INTRODUCTION
Continual learning aims to develop models capable of acquiring and retaining knowledge from sequence of tasks or data distributions, thereby mimicking the human ability to learn progressively over time. This approach is also known as lifelong learning incremental learning Thrun (1995) and holds promise for building adaptive and efficient systems in dynamic cavironments. A core challenge in continual learning is catastrophic forgetting McCloskey & Cohen (9899) phenomenon where the model's performance on previously acquired tasks degrades significantly when updated with new data De Lange al. (2021) Parisi al. (2019); Masana et al. (2023)
Recently, various approaches have been proposed to mitigate catastrophic forgetting Bonicelli et al. al. (2024b). These approaches be broadly methods Lopez-Paz & Ranzato (2017); [Bang al. (202I); Van de Ven al. (2020); HHaes et al. (2020). Regularization-
(2022): Tong al. (2022); Qiao al. (2024)
categorized into three primary types Masana et al. ): Rehearsal-bbat
(2018): Li et al. based methods Kirkpatrick et al. (017)); Wołczyk et al. (2022); ;; Schwarz et al.
"Equal contribution. Both authors were graduate students under the supervision of Prof. Q. Yu. Corresponning author.

---

Published as a conference paper at ICLR 2026
(a). Richness of Synaptic Connections
(b). Multiple Synapses Connect Neurons
time
SNN
ANN
W
0.8
W
-1.3
Σ
N
Σ Σ
1.4
-0.5
(c). Modulation Based Eligibility Trace
(d). Selective Activation and Inhibition of Synapses
Spike Sequence
Flighility Trace
Tassh
Modiasiion Factor ( (oot)
Modulations
Moaul 2
Yrank
D
Maue <
0
Low
High D
Ifaee
Task
Unnsl 2
Eligibility Strougth
Duta
Encoding
Task T-1
Task T
Classifier
Figure 1: Overview of the MSCN framework. (a) The phenomenon of multi-synaptic connections in biological neurons. (b) Modeling of neurons with multiple synaptic connections in both SNNs and ANNs. (c) Local synaptic activity plasticity based on eligibility traces. Depending on the eligibility trace, synapses undergo potentiation (P), depression (D), remain unchanged (O), reflecting the modulation's 's strength and direction. (d) The process of selectively activating task-relevant synapses and inhibiting irrelevant ones.
(202cc), and Architecture-based methods Yoon et al.
(2017): Zhou et al. (2021) : Li et al. (2019);
Wang et al. (2022a); [Serra et al. (2018); Hung et al. (2019); Kang al. (2022a). Among these
three types, Architecture-based methods are particularly notable for the ability to dynamically adjust the network structure accommodate new tasks. They primarily rely on two strategies: network expansion and pruning. Expansion strategies Hung et al. (019); Li et al. (2019); Yoon et al. (2017) start with small model and dynamically expand the network to mitigate forgetting. In contrast, pruning strategies Maallya et al.] (2018); Wang et al. (2022b); Kang et al. (2022a) assign sub-network each old task. These sub-networks are pruned from pre-allocater allocated dense model. Further training restricted to the unpruned parameters only. While these strategies offer several advantages, including performance and the potential for forget-free learning some cases, they still face notable challenges: (i) Dynamic expansion requires continuous network growth to accommodate it hardware-une cendwaarr-uffriedlyyy. (ii) Both expansion and pruming methods are often sensitive to the task order, leading to significant performance variation depending the sequence of tasks.
tasks, which makes
Remarkably, the brain achieves continual learning without suffering from dynamic structural growth Rasch & Born| (207):; Joseph & Gu (202T), highlighting the potential of biologically inspired mechanisms as significant alternatives to artificial network expansion. Among these, multi-synaptic connectivity is believed to be key factor in supporting continual learning Shi al. (2025): Wu & Mell (2009); Ko et al. (20TT), as its diverse synaptic architecture enhances the information representation capacity of neural circuits. Meanwhile, Three-factor learning rules Frrmauxx Gerstner (2016);
Gerstner et al. (2018) have been widely studied as biologically plausible models of synaptic plasticity. which synaptic changes are not only modulated by global neuromodulatory signals but also depend on the local synaptic activity. Inspired by this, we propose MSCN, a novel framework that enhances representational flexibility and robustness by employing the multi-synaptic cooperation mechanism, rather than increasing network depth, width, or dynamically expanding the architecture, illustrated in Fig. I Our framework consists of two components: the multi-synapse connectivity structure that augments the model's representational capacity within fixed network architecture, and the synaptic plasticity modulation mechanism based on local synaptic activity. Such local synaptic modulatory signals to synaptic weight
activity is integrated via eligibility traces, which
2

---

Published as a conference paper at ICLR 2026
During learning, task-relevant synapses a dynamically selected, while irrelevant ones suppressed, thereby effectively minimizing interference across tasks.
updates.
Our contributions are as follows:
We propose MSCN, the first continual learning framework that explicitly leverages the multi-synaptic cooperation mechanism, providing biologically inspired and capacity- efficient solution. By maximally harnessing synaptic resources, MSCN unlocks the potential of fixedccapacity networks and substantially boosts the scalability of continual learning systems.
We design a modulatory mechanism based on local synaptic actívities that modulates synaptic plasticity through eligibility traces, enabling precise, setivity-dependee modulation at the synaptic level. This modulation significantly strengthens the robustness of continual learning models to task order variations, ensuring stable performance even under highly dynamic and unpredictable training sequences.
Extensive experiments on four benchmark datasets across both spiking and non-spiking architectures demonstrate that MSCN consistently outperforms state-of-the-an continual learning methods in terms of accuracy, forgetting mitigation, and robustness to task order, while also exhibiting competitive computational efficiency.
2 RELATED WORK
Continual Learning methods are roughly divided into three categories: Rehearsal based methods store past experiences in memory to mitigate forgetting. Some works Rebuff al. ) Tiwari et al. (2022); Zhou et al. (2022b); Jeeveswaran et al. (2023) design sampling strategies to allocate a limited memory budget, while others Lin et al. (2022). Rolnick et al. (2019); Sun et al. (2023) build special subspace of old tasks as the memory. Regularization--uuse methods aim to consolidate previous knowledge by adding extra regularization terms to the loss function. Some works Li & Hoiem (017)) Kirkpatrick
al. (2017): Cha et al. (2020) constrain important weights in the parameter space Akyürek et al. (2021) Rudner et al. (2022): Kim et al. (2023), feature representations Gao al. (2023). output logits Li & Hoiem (2017); Oh et al. (2022) to
al. 2022 ; Jeeveswaran
remain close to those of the old model. Architecture-bac based methods dedicate different incremental model structures towards each task minimize forgetting Zhou et al. (2022a); Lu et al. (2024); Kang ct al. (2022a). Some works /Sera et al. (2018); ; Yoon et al. (2019) Hu et al. (2023) adopt modular architectures by dynamically expanding additional components Yan et al. (2021); x Zhu al. (2022). or freeze subsets of parameters AAbat et al. (2020); [Liu et al. (2021) to overcome forgetting. In this work, to better investigate the capacity efficiency and robustness of our method, we implement our multi-synaptic cooperation mechanism based on the mechteectur--bsee methods without dynamically expanding the network.
Neural Network Dynamies in continual learning describe how internal representations and connec- tivity patterns evolve as new tasks are learned sequentially Márton et al. (2022). These dynamics manifest across multiple levels, including synaptic updates, activation trajectories, and parameter plasticity under task transitions Vyas et al. 2002).. Recent works have begun to explicitly model neural dynamics in continual learning by introducing mechanisms such as synaptic trajectories, context-dependent modulation, and task-driven weight routing Li & Wang (2017): Li et al. (2024a); Xu et al. (2024). These studies highlight the importance of capturing temporal evolution in network parameters to support adaptive behavior over extended task sequences. Additional research explores dynamic mechanisms such as gating, masking, and pparsity-inducing priors modulate parameter updates and isolate task-specific pathways Abbai et al. (2020); Waan et al.) (2022c); Yan et al. (2022). Recently, multi-synaptic (redundant) connections between neuron pairs have been shown to enhance computational capacity Zenke & Laborieux (2024); Hofmann et al. (2025). Nevertheless, the coopera- tive interactions among multiple synapses and their implications for continual learning remain largely underexplored. In contrast, our method introduces multi-synaptic dynamics within each connection and the modulation based on local synapse activity, enabling representational diversity and adaptive modulation of synaptic plasticity. This novel design does not require increasing network depth/width or dynamically expanding the network: instead, it increases capacity and enhances robustness through multi- syntiiicyapiic cooperation mechanism.
3

---

Published as a conference paper at ICLR 2026
3 METHOD
3.1 MODELING MULTI-SYNAPTIC SPIKING NEURON
Biological systems achieve continual learning without relying on architectural growth [Song al. (2024): Shi et al. (2025). A key neurobiological observation is the presence of multiple synaptic contacts between the same axon-dendrite pair, providing redundancy and adaptability Trachtenberg et al. )2002;; Yang et al. (2014). Since our design operates at the synaptic level, the proposed method can be applied to both ANN and SNN Maass (1997); Gütig & Sompolinsky (2006): Yu et (2025) architectures. To better capture biological principles, we first model the multi-ynapticc cooperation mechanism SNNs, which are more consistent with biological processes and offer event-driven, temporally sparse, and energy-efficient computation Gütig & Sompolinsky (2006). As concrete instantiation, we adopt the leaky integrate-anc and-ffre (LIF) neuron Lapicque (1907 a widely used model balancing biological plausibility and computational simplicity Shiu et al. (2024), Brand &
Petruccione 2024). The membrane potential (it) evolves over continuous time as follows:
dv di
(V(() -vV(1) - Vress) + I(t)
where Tm is the membrane time constant, Vreat is the resting potential, and [(t) denotes the total synaptic input current. This formulation captures leakage and current integration but assumes a single synapse per connection, limiting diversity. To address this, we generalize the neuron model by introducing P parallel synapses for each synaptic connection. Consider a neuron with N presynaptic neurons, each forming P distinct synaptic pathways to it. Therefore, in continuous time, membrane potential of the postsynaptic neuron given by:
V(I) ΣΣ Wip PS 9 Σ
where Wip denotes the synaptic weight of the p-th parallel synapse associated with the i-th presynaptic neuron, PSPip represents the postsynaptic potential of this synapse and 9 denotes the firing threshold. To preserve synaptic heterogeneity Deng et al. (2025) and enable independent optimization, introduce distinct decay time constants across parallel synapses. Accordingly, for the p-th synapse of presynaptic neuron i, the spike arrival times are denoted by t.y. and PSP ip is defined as:
PSP (pt) = Σ Kip (t-/hh
(tp(t) =
(4)
where (t)(t) denotes the kernel function of the p-th parallel synapse, and Tes, represents its decay time constant, which is initialized to different values (non-trainatee This design allows multiple temporal and weighted channels to influence the synaptic plasticity. thereby enhancing the diversity of spatiotemporal representations. On this basis, modeling in the ANN architecture can be more readily formulated. Given space limitations, the corresponding implementation is presented in Appendix A.
3.2 PLASTICITY MODULATION BASED ON ELIGIBILITY TRACES
Building on multi-synaptic connections, we introduce a modulation mechanism of synaptic plasticity based on local synaptic activity. Specifically, we propose the eligibility trace as the basis for modulating local synaptic plasticity. We begin by formulating the modulation signal in continuous time as a cumulative sum synaptie spike events. For a connection between two neurons, P parallel synapses share a common eligibility trace, which is defined as:
de
5 (t-tt/)
dt
4

---

Published as a conference paper at ICLR 2026
where is the decay time constant, 8(-) denotes the Dirac delta function. For practical implementation, we adopt a discrete-time formulation, and the dynamics of are updated as follows:
e[d + +]
ēlt + 1| a|d
(6)
where Sit + 1] E { indicates whether spike occurred at time step t + 1. This design captures recent spiking accumulation while allowing the eligibility trace to decay in the absence of input, enabling the eligibility trace to track local synaptic activity over time.
The effect of the modulation factor on synaptic plasticity is governed by a nonlinear function fmod (ē). which determines how strongly and in what direction the synapse should change in response to local synaptic activity signals. Following (Zhang et al. (2023), we adopt a piecewise quadratic form for depression) of synaptic plasticity
fmod, which adjusts the strength and direction (potentiation depending on the eligibility trace. practice, é is normalized over all model synapses to [-1,1] reflect the relative strength of local synaptic activity. The modulation function is defined as:
2
E
+ (0max - 1)
C1
12
61
2
fmod (e)
Gmax
61
Cinv:
(7)
Cinv - E1
(mnod(2zuu
(ē))
Cinv S |ē 20
0,
otherwise
where e 0.5 einv sets the low-moderate boundary, Omax controls the maximum modulation strength, and Cinv marks the zero point of fod (Fig. [b). This function modulates plasticity based on local synaptic activity: potentiation occurs when |modd 2 1, depression arises when |/moal < 1, and complete depression is observed when fmod = 0. The final modulated synaptic weight change is computed as:
OC
A -1 fmod (c)
(8)
Ow
where y7 is the learning rate and aL/Ow is the gradient. The modulation function fiod adjusts the size and direction of the synaptic weight update. By dynamically adjusting synaptic updates in response to recent local activity, this design mimics the biological mechanism of robust learning through activity-dependent modulation Wu et al. (202T); Wu & Maass (2025), thereby enabling the network to adapt to changing task demands and maintain stable performance.
3.3 GENERALIZING TO CLASSIC ARCHITECTURE BASED METHODS
In this section, we integrate our method into the Architecture setting, adopting the same setup as in Kang et al. 2022b); Serra et al. (2018); Wortsman et al. (2020). Consider a standard supervised continual learning setting with 7 tasks presented sequentially. For each task j. the model receives a dataset D, { yr.s) ) consisting of nj labeled samples. fixed-topology deep neural network F-; 0), parameterized by model parameters 0, is employed. The objective each step is to optimize the model for the current task j:
n n, Σ (FF(x);:)))
0" minimize
Following Gao et al. 2023): Wortsman et al. 2020), task identities are assumed to be available during both training and inference, under multi-head setting in which each task is assigned distinct output head. For each task j. a binary mask m is learned to activate the relevant synapses. The training objective is formulated
5

---

Published as a conference paper at ICLR 2026
Rj
0", m3 minimize
1 f
( i 0 e my). (x 0).
(10)
0
where denotes element-win multiplication. A shared learnable relevance score r is maintained across tasks, with each entry corresponding to synapse Kang al. (2022b). Trained jointly with the network parameters, r enables the model to identify task-relevant connections. For task j. the subnetwork 02 is formed by selecting the top c% of weights ranked by relevance, where c is the layerwise capacity ratio Wortsman al. (2020). The selected weights are indicated by the binary mask m, in which a value of signifies that the corresponding weight active during the forward pass, and 0 indicates is deactivated. To preserve past knowledge, an accumulated mask M,-1- = Vi=i m, (with V as logical OR) applied when learning task j. The parameters updated as:
θ- 0
A
MG-1)
(11)
where AO denotes the gradient step, and the term (1 M₃-1) ensures that only unallocated synapses remain trainable, thereby preserving the stability of parameters from previously learned tasks.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
We conduct comprehensive experiments under diverse training configurations, input domains, datasets, and network architectures, and evaluate continual learning performance using two widely adopted metrics Kang et al. (2022a): : Konishi et al. (2023): Average Accuracy (ACC) and Backward Transfer (BWT). ACC measures the model's average accuracy over all tasks after learning task T, reflecting its overall generalization ability. BWT measures the impact of new tasks prior ones, with higher values better and indicating no forgetting. Unless otherwise stated, we set the synapse count to P = 3 in all main experiments, ablation studies, and computational cost analyses. Implementation and hardware details are provided in Appendix A.2
Table 1: Performance comparison on four datasets, evaluating performance under both SNN and ANN frameworks. We report the results 5 independent runs with different random seeds under the same experimental setup. Table in Appendix A.2 shows the standard deviations.
Network Method
PMNIST
10-split CIFAR-100
TinylmageNet
S-Daiisets
(S)
BWT (8))
ACC (%)
BWT (%)
ACC (%)
BWT (16
ACC
BWT (%6)
4F7L.
96.52
79.83
79.24
89.93
Kirkyurrick
-3.20
73.75
44..5
SNN
HATEEM Strarel
20TS
93.25
-2.07
73.67
-0.13
62.18
-8.51
72.72
-22.90
GPM
2021
94.50
77.48
-1.37
70.07
-2.92
79.70
-15.52
Xu
-0.26
-3.71
MSCN
96.34
0.0
79.54
0.0
73.22
0.0
0.0
EWC Kirkpolric
2017
92.01
-0.03
72.77
-3.59
64.51
-0.01
$3.64
-0.4
GPM
+1.17
PackNat
Millya al
2018
96.37
0.0
72.
0.0
55.46
0.D
92.81
0.0
SupSt
Weehsmon
2020
96.31
0.0
75.47
0.0
59.60
93.28
0.0
WSS
Clell
ANN
TAMIL
Bital ar
96.87
-3.15
76.73
+3.47
72.55
-3.02
93.47
-472
SPG
Konach
96.35
74.82
73.26
93.32
DFGPle
Buyesian"
Iappa &
0024
96.74
75.57
73.93
93.36
MSCN
97.53
0.00
77.37
0.0
75.03
0.0
93.69
0.0
4.2 COMPARISON TO THE STATE-OF- OF- METHODS
We first conduct comprehensive evaluation MSCN in a multi-head task- iakkinnceeentall learning scenario, using four widely used benchmark datasets and employing both SNNs and ANNs. As summarized in Table [] where the dataset complexity roughly increases from left to right, the results consistently demonstrate the strength and reliability of MSCN across diverse datasets and

---

Published as a conference paper at ICLR 2026
EWC
WSN
TAMiL
SPG
MSCN
(a) Permuted MNIST
(b) 10-split CIFAR-100
(c) 5-Datasets
100
100
Accuracy (%)
98
80
98
96
96
75
94
94
70
92
92
90
10 Increasing No. of Tasks
2
3
4
8 9
65
2
6
8 9
10
90
2
Increasing No. of Tasks
Increasing No. of Tasks
Figure 2: Per-task accuracy across the entire incremental learning.
architectures. Notably, on TinylmageNet with ANN, MSCN achieves an accuracy that exceeds the second-best method, Bayesian Thapa & Li (2024) by 1.10%. Although several approaches, such as WSN Kang et al. (2022a). SupSup Wortsman et al. (2020), and PackNet Mallya et al. (2018), achieve zero backward transfer, they do match the overall accuracy of MSCN. These findings indicate that muuli--ynaaptic connectivity is an effective and scalable design for continual learning. Moreover, Fig. 2 shows that MSCN attains superior per-task performance on most tasks, further validating its representational strength. Since most existing works on continual learning are based on ANN architectures, we investigate our method extensively within the same framework in the following sections to ensure fairness while highlighting its strengths.
4.3 ROBUSTNESS TO TASK ORDER PERMUTATIONS
We evaluate order robustness by training on multiple CIFAR-100 Split permutations and measuring per-task accuracy variation three task orders. As illustrated in Fig.Bi. we observe that EWC patrick et al. (2017) and GPM Saha et al. (2021) display large fluctuations across task sequences, highlighting their strong sensitivity order. WSN Kang et (2022a) performs competitively with EWC but shows a tendency to overfit to particular task orders (Fig. Ba-b). In contrast, MSCN achieves stable accuracy across all tasks and permutations, with only minimal variation (Fig. 3+). This consistency indicates that MSCN can flexibly adapt to new tasks while mitigating interference and preserving prior knowledge, highlighting the important modulation as foundational mechanism for building scalable continual learning systems. Additional experiments on five task orders are reported in Fig. of of the Appendix A.3.4
of local
synaptic
(a) BWC
(b) WSN
(c) MSCN
(d) Variance of Each Task
30
80
EWC GPM
Accuracy (%)
Varanne
WSN
40
40
40
12
MSONN
20
20
20
10
Task Index
Task Index
Task Index
Task Index
Figure 3: Task order robustness comparison on CIFAR-100 Split. Bar plots in (a), (b), and (c) show per-task accuracy under three different task sequences. Specifically, (a) corresponds to the EWC method, (b) the WSN method, and (c) to our MSCN method. (d) shows the standard deviation of the accuracy for each task across different task sequences.
7

---

Published as a conference paper at ICLR 2026
4.4 ABLATION STUDY
To better understand the individual contributions of the key components in MSCN, we perform an ablation study by selectively removing the multi-synaptic connectivity structure and the neuromod- ulatory mechanism. The results across four benchmark datasets are summarized in Table 2 The full MSCN model. with both components enabled, consistently achieves the highest accuracy across all datasets. Disabling the neuromodulatory mechanism results in noticeable performance drop, particularly on IinylmageNet and PMNIST. Conversely, removing the multi-synaptic structure also results in performance degradation, particularly on CIFAR- 1-0 and 5-Datasets. We observe that when both components are ablated, performance drops further across all datasets, confirming that the two mechanisms act cooperatively and that their interaction is essential for MSCN's ability to achieve robust and scalable continual learning.
Table 2: An ablation study of MSCN on ACC. Vindicates that the component is included, while X indicates that is excluded.
Multi synape
Modulation
PMNIST
CIFAR-100
TinylmageNet
5-Datasets
97.53 (#: 0.19)
77.37 (# 0.23)
75.03 (#: 0.27)
93.69 (#: 0.21)
96.79 (1: 0.20)
77.03 (#: 0.22)
73.81 (#: 0.26)
93.47 (4: 0.24)
X
x
96.53 (#: 0.21)
76.81 (# 0.25)
73.78 (# 0.29)
93.51 (# 0.22)
x
96.34 0.22)
76.34 (# 0.24)
72.59 (:: 0.31)
93.32 (E: 0.25)
4.5 EFFECT OF SYNAPSE COUNT
To probe the role of multi-synaptic connectivity, vary synapse and neuron counts and measure performance across tasks. On CIFAR- Split (Fig. 4.. increasing synapses per connection consis- tently raises per-task accuracy the number of tasks grows. Jointly scaling synapses and neurons (Fig. 55) reveals a saturation regime: accuracy improves with capacity but plateaus once model capacity exceeds task complexity. Fig. 6 summarizes four benchmarks; rows (top->bottom) PMNIST, CIFAR-100, TinylmageNet, and D-Dataett, reflecting increasing complexity. We observe that, although the optimal number of synapses varies with task difficulty, it generally stabilizes once certain threshold is exceeded. Interestingly, the observed performance trend happens to mirror how synaptic counts are distributed in the trainnlypically confined a limited but effective range Toni Additional experimental results are provided in Appendix A.3.3
et al.
1999): Watson et al. 2025)
0.2 605
2.11
6/5
3.00
Nomallaed ACC
Aecrage Aduracy
Alco
0.6
0.4
0.2
0.38
0.20
337
335
009
88
1.09
0:02
1.00
Synapsos
4
Mumber Synapses
Figure 4: Per-task accu- racy under different synapse counts on CIFAR-100 Split.
Figure Accuracy under
Figure 6: Accuracy changes across four datasets under varying num- bers of synapses
different synapse and neuron counts on CIFAR-100 Split.
4.6 CAPACITY ANALYSIS
To evaluate the capacity efficiency of MSCN, we adopt the commonly used metric CAP CAP et al. (2022a); Wortsman al. (2020) (defined in Appendix A.2..) and compare it with baseline approaches, where lower CAP values indicate better capacity utilization. Fig. shoows the relationship between accuracy and total capacity usage across four benchmark datasets. The results demonstrate that MSCN
8

---

Published as a conference paper at ICLR 2026
consistently achieves the highest accuracy with significantly lower capacity overhead. On Permuted MNIST and TinylmageNett (Fig. 7.c). MSCN outperforms all baselines while requiring substantially fewer resources. Similar trends are observed where methods such as PackNet Mallya et al. (0018) and SupSup Wortsman et al. (2020) consume much more capacity but yield lower or comparable performance. Thhsse results indicate that the multi-synaptic cooperation mechanism enables more effective knowledge storage and reuse across
CIFAR- 100 and the 5-Datasets benchmark (Fig. (),
tasks.
(a) Permuted MNIST
(b) 10-split CIFAR-100
(c) TinyimageNet
(d) 5-Datasets
Suptiup
77
94-
ACC (%6
76
92-
74
90
93
91
100
120
117
227
104
109
(%)
(%)
Figure 7: Accuracy over CAP (%) across four benchmarks.
4.7 COMPUTATIONAL EFFICIENCY
Since multi-synaptic connections inevitably increase the parameter count, conducted experiments under the same parameter budget as the baselines to fairly evaluate computational efficiency. We further evaluated robustness under five task orders using AOPD (where lower values indicate stronger robustness; see Appendix A.2.2 for details). As shown in Table 3 although reducing the parameter count leads drop in MSCN's accuracy, method consistently achieves the best efficiency and robustness while maintaining competitive accuracy. These results demonstrate that the proposed multi-synaptic cooperation mechanism achieves high computational efficiency and establishes the foundation for robust and scalable continual learning.
Table 3: Computational efficiency under the same parameter budget (Training time in hours).
PMNIST
10-split CIFAR-100
Method
Training Time
ACC (%)
AOPD (%)
Training Time
ACC (%) f
AOPD (%) |
PackNet
0.59 (#: 0.15)
96.43 (# 0.18)
2.23
1.13 (t: 0.10)
72.45 (1: 0.20)
5.36
SupSup
0.53 (# 0.12)
96.36 (# 0.22)
1.27
0.87 (# 0.08)
75.54 (#: 0.17)
3.81
WSN
0.38 (1: 0.05)
96.49 (: 0.13)
0.29
0.78 (1: 0.06)
76.47 (1: 0.34)
2.59
MSCN
0.33
96.89 0.19)
0.23
0.65 0.04)
76.40 (1
2.41
TinyImageNet
5-Datasets ACC (%) f
Method
Training Time +
ACC (%) f AOPD (%)
Training Time
AOPD (%) 4
PackNet
1,45 (# 0.12)
55.51 (# 0.25)
6.51
3.45 (t 0.08)
92.89 ( 0.12)
4.37
SupSup
0.97 (# 0.07)
59.65 (# 0.24)
6.94
3.26 (+ 0.10)
93.31 ( 0.16)
2.83
WSN
0.92
72.03 0.41)
4.98
3.05 (+ 0.08)
93.50 0.13)
1.37
MSCN
0.75 0.03)
74.04 (= 0.21)
4.73
2.82 (t: 0.09)
93.33 (t 0.09)
1.26
CONCLUSION
In this paper, we propose MSCN, the first continual learning framework that explicitly models multi- synaptic cooperation. By equipping each connection with multiple plastic synapses and employing local synaptic activity-based modulation, MSCN achieves effective knowledge retention and adapt- ability within fixed architecture. Extensive evaluations across diverse datasets and architectures demonstrate that our approach consistently outperforms state-of-ther oof-the-art baselines in terms of compu- tational efficiency and task order robustness. These findings highlight that the synergistic interplay

---

Published as a conference paper at ICLR 2026
between multi-synaptic connectivity and localized plasticity modulation substantially enhances the network's ' representational capacity, providing new insights for robust and scalable continual learning. In addition, the similarity between our model's synapse counts and those found in biological systems further supports the plausibility of our design. Future work will explore allocating different numbers of synapses across neuron connections to further optimize synaptic resource utilization.
ACKNOWLEDGMENTS
This work in part supported by the Brain Science and Brain-like Intelligence Technology National Science and Technology Major Project (Grant No. 20252D0215600, the National Natural Science Foundation of China (Grant Nos. 92370103 and 62176179), and in part by the Xiaomi Foundation. We thank the reviewers for their insightful and supportive comments.
REFERENCES
Davide Abati, Jakub Tomezak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, and Babak Ehteshami Bejnordi. Conditional channel gated networks for task-aware continual learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3931-3940, 2020.
Afra Feyza Akyürek, Ekin Akyürek, Derry Tanti Wijaya, and Jacob Andreas. Subspace regularizers for few-shot class incremental learning. arXiv preprint arXiv:2/10.07000 2021.
Jihwan Bang, Heesu Kim, YoungJoon Yoo, Lung-Woo Ha, and Jonghyun Choi. Rainbow memory: Continual learning with a memory of diverse samples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6218-8227, 2222,20211 2021.
Prashant Bhat, Bahram Zonooz, and Elahe Arani. Task-aware information routing from common representation space in lifelong learning. arXiv preprint urXiv:2302./1346, 2023.
Lorenzo Bonicelli, Matteo Boschini, Angelo Porrello, Concetto Spampinato, and Simone Calderara. On the effectiveness of lipschitz-driven rehearsal in continual learning. Advances in Neural Information Processing Systems,
31901, 2022.
Dean Brand and Francesco Petruccione. A quantum leaky integrate-and-fire spiking neuron and network. upj Quantum Information, (1):125, 2024.
Alexander R Callan, Martin HcB, Felix Felmy, and Christian Leibold. Arrangement of excitatory synaptic inputs dendrites of the medial superior olive. Journal of Neuroscience, (2):269-283, 2021.
Sungmin Cha, Hsiang Hsu, Taeback Hwang, Flavio P Calmon, and Taesup Moon. Cpr: Classifier- projection regularization for continual learning. arXiv preprint Stti:200060032266 2020.
Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Tiny episodic memories in continual learning. In International Conference on Machine Learning, pp. 1286-1295.
PMLR, 2019.
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. continual learning survey: Defying forgetting classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7):3366-3333 2021.
Bohan Deng, De-An Huang, Li Fei-Fei, and Juan Carlos Niebles. Modular continual learning in unified visual environment. In European Conference on Computer Vision (ECCV), 2021.
Zhichao Deng, Zhikun Liu, Junxue Wang, Shengqian Chen, Xiang Wei, and Qiang Yu. Hetsyn: Versatile timescale integration in spiking neural networks via heterogeneous synapses. In The Thirty-ninth Annual Conference on Neural Information P Processing Systems, 2025.
Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. Orthogonal gradient descent for continual learning. In International conference on artificial intelligence and statistics, pp. 3762-3773. PMLR, 2020.
10

---

Published as a conference paper at ICLR 2026
Nicolas Frémaux and Wulfram Gerstner. Neuromodulated teaeee-iiinn-eepppee plasticity, and theory of three-factor learning rules. Frontiers in neural circuits, pp. 85, 2016.
Qiang Gao, Xiaojun Shan, Yuchen Zhang, and Fan Zhou. Enhancing knowledge transfer for task incremental learning with data-free subnetwork. Advances in Neural Information Processing Systems, pp. 68471-68488 2023.
Qiankun Gao, Chen Zhao, Bernard Ghanem, and Jian Zhang. R-dfcil: Relation-guidee representation learning for data-free class incremental learning. European Conference on Computer Vision, pp. 423-439. Springer, 2022.
Wulfram Gerstner, Marco Lehmann, Vasiliki Liakoni, Dane Corneil, and Johanni Brea. Eligibility traces and plasticity on behavioral time scales: experimental support of neohebbian three-factor learning rules. Frontiers in neural circuits, pp. 53, 2018.
Aayush Gupta, Vinay Verma, and Piyush Rai. Bias mitigation in continual learning using adaptive task balancing. International Conference on Learning Representations, 2020.
Robert Gütig and Haim Sompolinsky. The tempotron: a neuron that learns spike timing-based decisions. Nature neuroscience, (3):420-428, 2006.
Tyler L Hayes, Kushal Kafle, Robik Shrestha, Manoj Acharya, and Christopher Kanan. Remind your neural network to prevent catastrophic forgetting. In European conference on computer vision, pp- 466-483. Springer, 2020.
Martin Hofmann, Moritz Franz Peter Becker, Christian Tetzlaff, and Patrick Mäder. Concept transfer of synaptic diversity from biological to artificial neural networks. Nature communications, (1): 5112, 2025.
Zhiyuan Hu, Yunsheng Li. Jiancheng Lyu, Dashan Gao, and Nuno Vasconcelos. Dense network expansion for class incremental learning. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1
2023.
Ching Yi Hung, Cheng-Hao Tu, Cheng-En Wu, Chien-Hung Chen, Yi-Ming Chan, and Chu- Chu-Song Chen. Compacting, picking and growing for unforgetting continual learning. Advances in neural information processing systems, 32, 2019.
Kishaan Jeeveswaran, Prashant Bhat, Bahram Zonooz, and Elahe Arani. Birt: Bio-inspired replay in vision transformers for continual learning. arXiv preprinl 2rXiv:2305.04769 2023.
Joel Joseph and Alex Gu. La-maml: Look-ahead meta learning for continual learning, ml repro- ducibility challenge 2020. 2021.
Haeyong Kang, Rusty John Lloyd Mina. Sultan Rizky Hikmawan Madjid, Jachong Yoon, Mark Hasegawa-Johnson, Sung Ju Hwang, and Chang D Yoo. Forget-free continual learning with winning subnetworks. In International conference on machine learning, pp. 10734-10750. PMLR, 2022a.
Haeyong Kang, Jachong Yoon, Sultan Rizky Hikmawan Madjid. Sung Ju Hwang, and Chang D Yoo. On the soft-subnetwork for few-shot class incremental learning, arXiv preprint arXiv:2209.07529, 2022b.
Do- Yo-Yeon Kim, Dong-Jun Han, Jun Seo, and Jackyun Moon. Warping the space: Weight space rotation for class innremmaace few-shot learning. In The Eleventh International Conference on Learning Representations, 2023.
James Kirkpatriek, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Bann Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114 (13):3521-3526, 2017.
Ho Ko, Sonja B Hofer, Bruno Pichler, Katherine A Buchanan, Jesper Sjöström, and Thomas D Mrsic-Flogel. Functional specificity local synaptic connections in neocortical networks. Nature, (7345):87-91, 2011.
11

---

Published as a conference paper at ICLR 2026
Tatsuya Konishi, Mori Kurokawa, Chihiro Ono, Zixuan Ke, Gyuhak Kim, and Bing Liu. Parameter- level mast-mas for continual learning. In International Conference on Machine Learning, pp- 17492-17505. PMLR, 2023.
Louis Édouard Lapieque. Louis lapieque. J. physiol, pp. 620-633-3 1907.
Depeng Li, Tianqi Wang, Junwei Chen, Wei Dai, and Zhigang Zeng. Hamnessing neural unit dynamics for effective and scalable class-incremental learning. arXiv preprini arXiv:2406.02428,, 2024a.
Depeng Li, Tianqi Wang, Junwei Chen, Kenji Kawaguchi, Cheng Lian, and Zhigang Zeng. Multi-view class incremental learning. Information Fusion, 102: 102021, 2024b.
Depeng Li. Tianqi Wang, Junwei Chen, Qining Ren, Kenji Kawaguchi. and Zhigang Zeng. Towards continual learning desiderata via hsic-bottleneck orthogonalization and equiangular embedding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 13464-13473, 2024c.
Ming Li and Dianhui Wang. Insights into randomized algorithms for neural networks: Practical issues and common pitfalls. Information Sciences, 382: 382:170-1 70-1788, 2017.
Xilai Li, Yingho Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: continual structure learning framework for overcoming catastrophic forgetting. In International conference on machine learning, pp. 3925-3934. PMLR, 2019.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions pattern analysis and machine intelligence, (12):2935-2947, 2017.
Sen Lin, Li Yang, Deliang Fan, and Junshan Zhang, Trgp: Trust region gradient projection for continual learning. arXiv preprint wrXiv:2202.029311 2022.
Yaoyao Liu, Bernt Schicle, and Qianru Sun. Adaptive aggregation networks for class- innremenntt learning. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 2544 2553- 2021.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Aojun Lu, Tao Feng, Hangjie Yuan, Xiaotian Song, and Yanan Sun. Revisiting neural networks for continual learning: An architectural perspective. arXiv preprint urXiv:2404.14829 2024.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models.
Neural networks, (9):1699-167, 1997.
Arun Mallya, Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting a single network to multiple tasks by learning to mask weights. In Proceedings of the European conference on computer vision (ECCV), pp. 67-82, 2018
Christian David Márton, Siyan Zhou, and Kanaka Rajan. Linking task structure and neural network dynamics. Nature neuroscience, 25(6):679-68) 2022.
Mare Masana, Xialei Liu, Bartlomiej Twardowski, Mikel Menta, Andrew D. Bagdanov, and Joost Van De Weijer. Incremeaall Learning: Survey and Performance Evaluation on Image Classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, (5)55513-5533, 2023.
Michael McCloskey and Neal Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Seyed Iman Mirzadeh, Arslan Chaudhry, Dong Yin, Huiyi Hu, Razvan Pascanu, Dilan Gorur, and Mehrdad Farajtabar. Wide neural networks forget less catastrophically. In International conference machine learning, pp. 15699-157177 PMLR, 2022.
12

---

Published as a conference paper at ICLR 2026
Youngmin Oh, Donghyeon Back, and Bumsub Ham. Alife: Adaptive logit regularizer and feature replay for incremental semantic segmentation. Advances in Neural Information Processing Systems, pp. 14516-14528, 2022.
German Parisi, Ronald Kemker, Jose Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural networks, 113:54-71, 2019.
Zhongzheng Qiao, Quang Pham, Zhen Cao, Hoang H Le, Ponnuthurai N Suganthan, Xudong Jiang, and Savitha Ramasamy. Class-incremental learning for time series: Benchmark and evaluation. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1613-5624, 2024.
Björn Rasch and Jan Born. Maintaining memories by reactivation. Current opinion in neurobiology, 17(6):698-700 703, 2007.
Sylvestre- Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances in neural information processing systems, 2019.
Tim GJ Rudner, Freddie Bickford Smith, Qixuan Feng, Yee Whye Teh, and Yarin Gal. Continual learning via sequential function-space variational inference. In International Conference on Machine Learning, pp. 18871-18887. PMLR, 2022.
Gobinda Saha, Isha Garg, and Kaushik Roy. Gradient projection memory for continual learning. arXiv preprint arXiv:2103.0973 097622 2021.
Jonathan Schwarz, Wojeiech Czarnecki, Jelena Luketina, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable framework for continual learning. In International conference machine learning, pp. 4528 4537. PMLR, 2018.
Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International conference machine learning, pp. 4548 4557. PMLR, 2018
Qianqian Shi, Faqiang Liu, Hongyi Li, Guangyu Li, Luping Shi, and Rong Zhao. Hybrid neural networks for continual learning inspired by corticohippocampal eircuits. Nature Communications, (1):1272, 2025.
Philip K Shiu, Gabriella R Sterne, Nico Spiller, Romain Franconville, Andrea Sandoval, Joie Zhou, Neha Simha, Chan Hyuk Kang. Scongbong Yu, Jinscop Kim, et al. A drosophila computational brain model reveals sensorimotor processing. Nature, (8032):210-211 2024.
Yuhang Song, Beren Millidge, Tommaso Salvatori, Thomas Lukasiewicz. Zhenghua Xu, and Rafal Bogacz. Inferring neural activity before plasticity foundation for learning beyond backpropa- gation. Nature neuroscience, (2):348-355 2024.
Wenju Sun, Qingyong Li, Jing Zhang, Wen Wang, and Yangli-ao Geng. Decoupling learning and remembering: A bilevel memory framework with knowledge projection for task-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20186-20195, 2023.
Jeevan Thapa and Rui Li. Bayesian adaptation of network depth and width for continual learning. In Forty-first International Conference on Machine Learning, 2024.
Sebastian Thrun A lifelong learning perspective for mobile robot control. Intelligent robots and systems, pp. 201-214. Elsevier, 1995.
Rishabh Tiwari, Krishnateja Killamsetty, Rishabh Iyer, and Pradeep Shenoy. Ger: Gradient coreset based replay buffer selection for continual learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 90-, 2022.
13

---

Published as a conference paper at ICLR 2026
Shengbang Tong, Xili Dai, Ziyang Wu, Mingyang Li, Brent Yi, and Yi Ma. Incremental learning of structured memory via closed-loop transcription. arXiv preprint arXiv:2202.05411, 2022.
Nicolas Toni, P-A Buchs, Irina Nikonenko, CR Bron, and Dominique Muller. Ltp promotes formation of multiple spine synapses between a single axon terminal and a dendrite. Nature, (6760):421 425,
1999.
Joshua T Trachtenberg, Brian E Chen, Graham W Knott, Guoping Feng, Joshua R Sanes, Egbert Welker, and Karel Svoboda. Long-term in vivo imaging of experience-deeendent synaptic plasticity in adult cortex. Nature, (6917):788-794 2002.
Marta Turegano-Lopez, Felix de Las Pozas, Andrea Santuy, Jose-Rodrigo Rodriguez, Javier DeFelipe, and Angel Merchan-Peree Tracing nerve fibers with volume electron microscopy to quantitatively analyze brain connectivity. Communications Biology, (1):796, 2024.
Gido M Van de Ven, Hava T Siegelmann, and Andreas S Tolias. Brain-inspired replay for continual learning with artificial neural networks. Nature communications, 11(1):4069, 2020.
Saurabh Vyas, Matthew D Golub, David Sussillo, and Krishna V Shenoy. Computation through neural population dynamics. Annual review of neuroscience, 43(1):249-233 2020.
Fu- Yun Wang, Da-Wei Zhou, Liu Liu, Han-Jia Ye, Yatao Bian, De-Chuan Zhan, and Peilin Zhao. Beef: Bi-compatible class-incremental learning via energy-based expansion and fusion. In The eleventh international conference on learning representations, 2022a.
Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained transformers:
An occam's 's razor for domain incremental learning. Advances in Neural Information Processing Systems, pp. 5682-5695, 2022b.
Zifeng Wang. Zheng Zhan, Yifan Gong, Geng Yuan, Wei Niu, Tong Jian, Bin Ren, Stratis Ioannidis, Yanzhi Wang, and Jennifer Dy. Sparcl: Sparse continual learning on the edge. Advances in Neural Information Processing Systems, pp. 20380, 2022c.
Jake F Watson, Victor Vargas-Bann Barroso, Rebecca Morse-Mora, Andrea Navve-Olive, Mojtaba R Tavakoli, Johann G Danzl, Matthias Tomschik, Karl Rössler, and Peter Jonas. Human hippocampal ca3 uses specific functional connectivity rules for efficient associative memory. Cell, (2):501-514, 2025.
Maciej Wolezyk, Karol Piezak, Bartosz Wójeik, Lukasz Pustelnik, Pawel Morawiecki, Jacek Tabor, Tomasz Trzcinski, and Przemystaw Spurek. Continual learning with guarantees via weight interval constraints. In International Conference on Machine Learning, pp. 23897-23911. PMLR, 2022.
Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in superposition. Advances in neural information processing systems, 33:15173-15184 2020.
Chi-Hong Wu, Raul Ramos, Donald B Katz, and Gina G Turrigiano. Homeostatic synaptic scaling establishes the specificity of an associative memory. Current biology, (11):2274-2285, 2021.
Xundong E Wu and Bartlett W Mel. Capacity-enhancing synaptic learning rules in a medial temporal lobe online learning model. Neuron, (1):31-41, 2009.
Yujie Wu and Wolfgang Maass. simple model for behavioral time scale synaptic plasticity (btsp) provides content addressable memory with binary synapses and one-shot learning. Nature communications, (1):342, 2025.
Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, and Zhouchen Lin. Hebbian learning based orthogonal projection for continual learning of spiking neural networks. arXiv preprint arXiv:2402. 199444442..194, 2024.
Mingkun Xu, Faqiang Liu, Yifan Hu, Hongyi Li, Yuanyuan Wei, Shuai Zhong, Jing Pei, and Lei Deng. Adaptive synaptic scaling in spiking networks for continual learning and enhanced robustness. IEEE Transactions Neural Networks and Learning Systems, 2024.
14

---

Published as a conference paper at ICLR 2026
Qingsen Yan, Dong Gong, Yuhang Liu, Anton Van Den Hengel, and Javen Qinfeng Shi. Learning bayesian sparse networks with full experience replay for continual learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 109-118. 2022.
Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVE conference computer vision and paltern recognition, pp. 3014-3023, 2021.
Enneng Yang. Shen, Zhenyi Wang. Shiwei Liu, Guibing Guo, and Xingwei Wang. Data aug- mented flatness-aware gradient projection for continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5630-5634 2023.
Guang Yang, Cora Sau Wan Lai, Joseph Cichon, Lei Ma, Wei Li, and Wen-Biao Gan. Sleep promotes branch-specific formation of dendritic spines after learning. Science, (6188):1173-11)) 2014.
Jachong Yoon, Eunho Yang. Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint wrXiv:1788.01441 2017.
Jachong Yoon, Sachoon Kim, Eunho Yang. and Sung Ju Hwang. Scalable and order-robust continual learning with additive parameter decomposition. arXiv preprint arXiv: 09432, 2019.
Qiang Yu, Misha Tsodyks, Haim Sompolinsky, Dietmar Schmitz, and Robert Gütig. Interactions between long-and short-term synaptic plasticity transform temporal neural representations into spatial. Proceedings of the National Academy of Sciences, 122(47))e242629012 22, 2025.
Friedemann Zenke and Axel Laborieux. Theories of synaptic memory consolidation and intelligent plasticity for continual learning. arXiv preprint arXiv:2405.16922, 2024.
Tielin Zhang, Xiang Cheng, Shuncheng Jia, Chengyu T Li, Mu-ming Poo, and Bo Xu. A brain- inspired algorithm that mitigates catastrophic forgetting of artificial and spiking neural networks with low computational cost. Science Advances, (34):aad, 2023.
Da-We Zhou, Qi-Wei Wang, Han-Jia Ye, and De-Chuan Zhan. A model or 603 exemplars: Towards memory-efficient class-incremental learning. arXiv preprint wrxivv2205.1321/1 2022a.
Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111.07832, 2021.
Xiao Zhou, Renjie Pi Weizhong Zhang, Yong Lin, Zonghao Chen, and Tong Zhang. Probabilistic bilevel coreset selection. In International conference machine learning, pp. 27287-27302 PMLR, 2022b.
Kai Zhu, Wei Zhai, Yang Cao, Jiebo Luo, and Zheng-Jun Zha. Self-sustaining representation expan- sion for non-exemplar class-incremental learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9296-9305, 2022.
A APPENDIX
A.1 MULTI-SYNAPTIC COOPERATION NETWORK IN ANNS
A.1.1 MODELING MULTI-SYNAPTIC NEURON
In biological neurons, a single axon can establish multiple synaptic contacts with the same dendritic branch, enabling the signal to influence the target neuron through parallel pathways TaraaaysTTrggaooooooooo ct al. (2024): Callan al. 2021). To simulate this structure in artificial neural networks, we represent each inter-neuronal connection not by single weight but by vector of parallel synaptic weights. Each element in this vector is trained independently, and their combined effect determines the connection strength. This design remains compatible with existing network architectures, supports standard gradient- based learning, and naturally extends plasticity rules developed for spiking neural networks.
15

---

Published as a conference paper at ICLR 2026
To model multi- synltisyyaaticc connections in ANNs, we first associate each connection (i.5) with column vector of synaptic weights way:
Waj
W
(12)
where a hyperparameter representing the number of synaptic connections between each pair of neurons. To preserve synaptic diversity and enable distinct optimization of parallel synapses, assign different activation functions to different parallel synapses. The input to postsynaptic neuron, from presynaptic neuron is then defined as:
(1((2;) = [u(
(13)
p=1
where oy denotes the activation function specific to the p-th synapse. When an input vector x is given to fully connected layer, the input to neuron is computed as
P ΣΣo i=1p=1
2 Σ g.y(
(14)
where N is the number of input neurons in the layer. The value žj is then passed through a ReLU activation function resulting the output yj max(0, 2j). Similarly, in convolutional layers, the additional synaptic dimension is incorporated into each filter. As result, the summation in Eq. (14)
also spans spatial locations and input channels.
For local synaptic plasticity modulation, we associate the local activity of each neuron with its output value. The eligibility trace is defined as follows:
a(t
ēlt + 1] = alt]
+
(15)
where r a decay time constant. Each synaptic weight is updated using the following learning rule:
ac Dw.y
Auyj = 11 [Smod (+))
(16)
where 17 is the learning rate. The eligibility trace ēg keeps track of local synaptic activity. The function fmod uses this information to smoothly adjust the synaptic strength strength-potentialee potentiation or de- cissiinn-smmilar to how synaptic plasticity is modulated in spiking neural models. When P Eq. [2. (T6) become the same as in standard ANNs, so this method remains fully compatible with existing implementations.
A.2 IMPLEMENTATION DETAILS
A.2.1 DATASETS.
The datasets used in our experiments are summarized Table 4 We evaluate our approach on four standard continual learning benchmarks: Permuted MNIST (PMNIST), CIFAR-100, and 5-Datasets, which are presented in roughly increasing order of dataset complexity.
PMNIST is variant of the original MNIST dataset consisting of 28 28 grayscale images of handwritten digits. Each task applies a fixed but unique random permutation to the pixel positions, making it widely adopted benchmark for evaluating robustness in maskinncrementa learning. For PMNIST, we assign each task 60,000 training and 10,000 testing samples to increase the challenge.
CIFAR-100. CIFAR-100 is object recognition dataset with 100 natural image classes. Following the protocol in Rebuffi et al. (2017). we partition the dataset into tasks, each comprising 10 disjoint classes with their corresponding images.
16

---

Published as a conference paper at ICLR 2026
TinyImageNet contains 100,000 64 64 color images across 200 classes. We construct 40 sequential tasks by splitting sample subset of the original dataset and align the test set with the training set as in Serra et al.
the dataset into 5-way classification problems. For fair comparison, we randomly
(2018).
5-Datasets combines tasks from five diverse datasets: MNIST, SVHN, Fashion-MNIST CIFAR-10, and NotMNIST, each treated as an independent task. This setting evaluates the model's s generalizabil- ity under distribution shift and cross-domain learning.
For CIFAR-100 and Tiny ImageNet, we follow standard settings with 500 training and 100/50 test images per class, respectively.
Table 4: Dataset statistics
Dataset
PMNIST CIFAR-100 TinylmageNet
5-Datasets
Tasks
40
5
Classes
100
200
Training Samples
60,000
50,000
100,000
Test Samples
10,000
10,000
10,000
/
A.2.2 EVALUATION METRICS
Following Kang et al. (2022a); Wortsman et al. (2020): Mallya et al. (2018), we evaluate all methods
based on the following metrics:
Accuracy (ACC) measures the average of the final classification accuracy on all tasks:
ACC
acct,
(17)
T
where acor, is the test accuracy for task i after training on task
Backward Transfer (BWT) measures the influence of learning tasks the performance of previously learned ones. A negative BWT indicates forgetting, whereas a positive BWT suggests that learning later tasks improved the performance of earlier ones. BWT is computed as:
T
BWT T-1
(aceT, acc,,))
(18)
where acc,, denotes the accuracy of task immediately after it is learned. A BWT close to zero implies stability, while highly negative values indicate catastrophic forgetting.
Capacity (CAP) measures the amount of network capacity used under each parameter pruning method Kang al. (2022a). It accounts for both the proportion of trainable parameters and the efficiency of binary encoding. The CAP metric is defined as:
CAP (1 - C)-C)+ (1 a)
(19)
32
where a the average mask compression rate (a=0.78), N is the number of tasks, and C' is the percentage of non-fixed parameters. A smaller CAP value indicates higher effective network capacity,
Average Order-normalized Performance Disparity (AOPD) measures the robustness of these al- gorithms under different task orders. Following the protocol of Yoon et al. (2019). assessed the task order robustness with the Order-normalized Performance Disparity (OPD) metric, which is computed the disparity between the performance A, of task t on R different task orders: OPD, e max(A),... ........ AR min A.). The average OPD (AOPD) is defined by
AOPD
A
T Σ OPD,
(20)
17

---

Published as a conference paper at ICLR 2026
Table 5: Performance deviations of the proposed method and baselines on four datasets.
(a) PMNIST and 10-spli CIFAR-100
PMNIST
10-split CIFAR-100
Network
Method
ACC (%)
BWT (%)
ACC (%)
BWT (%)
MTL
0.12
0.31
/
EWC Kirkpatrri et al.
2017
0.51
0.42
0.63
0.57
HAT Serra et al. 2018
0.24
0.0
0.37
0.19
SNN
GPM Saha et al. 2021
0.43
0.34
0.41
0.38
HLOP Xiao et al.
(2024
0.39
0.21
0.35
0.18
MSCN
0.22
0.0
0.25
0.0
MTL
0.14
0.21
/
SWCKkrkpttrick et al.
2017
0.56
0.01
0.57
0.49
GPM Saha et al. 2021
0.07
0.01
0.48
0.39
PackNet Mallya et al.
20188
0.04
0.0
0,41
0.0
ANN
SupSup Wortsman et al. (2020)
0.09
0.0
0.32
0.0
WSN Kang et al. (2022a)
0.07
0.0
0.29
0.0
TAMiL Bhat et al. 20033
0.17
0.04
0.36
0.49
MSCN
0.19
0.0
0.23
0.0
(b) TinylmageNet and 5 5-Datasets
TinyImageNet
5-Datasets
Network Method
ACC (%)
BWT (%)
ACC (%)
BWT (%)
MTL
0.29
0.26
/
EWC Kirkpatrrc et al. 2017
0.59
0.72
0.48
0.65
HAT Serra et al.
2018
0.61
0.44
0.33
0.51
SNN
GPM Saha et al. 2021
0.46
0.29
0.36
0.42
HLOP Xiao et al.
(2024
0.41
0.23
0.30
0.25
MSCN
0.31
0.0
0.29
0.0
MTL
0.33
0.27
/
EWC EiCkkaaatrrick et al.
2017
0.44
0.03
0.26
0.02
GPM Saha et al. 2021
0.42
0.31
0.20
0.01
PackNet Mallya et al.
2018
0.35
0.0
0.12
0.0
ANN
SupSup Wortsman et al.
(002
0.40
0.0
0.21
0.0
WSN Kang et al. 2202a
0.34
0.0
0.13
0.0
TAMiL Bhat et al. 2023
0.31
0.13
0.22
0.08
MSCN
0.27
0.0
0.21
0.0
A.2.3 EXPERIMENT SETTINGS
All experiments were conducted on Linux server equipped with Intel Xeon Gold 5220 (2.20 GHz)
CPU and two NVIDIA Tesla V100-SXM2 GPUs (32 GB each, driver 535. (55.1229.3) Following Kang two-layered MLP with 100
et al. (2022a); Wortsman al. 2020); Mallya al. (2018), neurons per layer for PMNIST and use modified version of AlexNet for the CIFAR- 100 Split dataset and reduced ResNet- 18 Chaudhry et al. (2019) Saha al. (201)) for 5-Datasets. For TinylmageNer,
18

---

Published as a conference paper at ICLR 2026
also use the same network architecture (uppa et al. (2020); Deng al. (2011) which consists of 4
Conv layers and 3 fully connected layers. For a fair comparison, we follow the experimental setting in Kang al. 2022a) Thapa & Li 2024). and all methods are evaluated under the same multi-head
setting with known task labels. The hyperparameter settings are presented in Table 7
Table 7: Experiment settings and hyperparameter configurations for different datasets
Datasct
PMNIST
10-split CIFAR-100
TinylmageNer
5-Datasets
learning rate
x 10-3
10-3
1 x 10-3
10-3
dropout rate
0.2
0.2
0.5
epochs batch size warmup ratio
5
200
10
100
10
64
10
64
0.05
0.05
0.05
0.05
optimizer
AdamW
Adam
Adam 1 x 10-22
AdamW
weight decay
x 10
x 10-2
1 x 10-2
architecture
Two-layered MLP
Modified AlexNet
Conv layers and Fully connected layers Reduced ResNet18
A.2.4 ARCHITECTURAL DETAILS
Two-layered MLP: In conducting the PMNIST experiments, we are following the exact setup as
denoted by Saha et al. (021) fully-connected network with two hidden layers of 100 neurons Lopez
Paz & Ranzato (2017).
Modified AlexNet: For the split CIFAR-100 dataset, we use a modified version of AlexNet similar to Joseph Gu (202T): Saha et al. (2021).
4 Conv layers and 3 Fully connected layers: For TinylmageNet, we use the same network architec- ture Joseph & Gu (202I) Deng al. (20TT.
Reduced ResNet18: In conducting the 5-Dataset experiments, we use a smaller version of ResNet18 with three times fewer feature maps across all layers as denoted by Lopez- Paz & Ranzato (2017).
A.2.5 LIST OF MAIN NOTATIONS
In T we list the main notations used in this paper, together with brief explanations, enabling quick reference to the meaning of each symbol.
A.3
EXTRA EXPERIMENTS
A.3.1 INTEGRATION WITH DIFFERENT TYPES OF METHODS
To further evaluate the applicability of our MSCN, integrated it with different types of methods and conducted additional experiments on both 10-split CIFAR-100 and TinyImageNet under the same parameter budget. As shown in Table 9 integrating MSCN into regularization-basse based methods consistently improves BWT. Notably, combining MSCN with ER on TinyImageNe improves BWT by 53.8% (an absolute decrease of 10.69). The observed BWT improvements attributed to the high capacity efficiency of MSCN, which arises from the independent optimization of multiple parallel synapses, as demonstrated in Fig. 7 Such higher capacity efficiency has been shown to reduce catastrophic forgetting Hung et al. 2019% Mirzadeh et al. (002)); Farajtabar et al. (2020). Meanwhile, the modulation mechanism further enhances this property by depressing the effect of noisy samples and strengthening learning on clean ones. In contrast, for architecture-base
and replay-
methods, BWT remains zero because the weights of past tasks are frozen, which is exactly as expected. At the same time, when our MSCN is incorporated. all three types of methods achieve improved accuracy. These additional experiments further highlight the robustness of our MSCN.
A.3.2 LAYER- WISE CAPACITY ANALYSIS
As shown in Fig. g CIFAR-100 Split, and TinylmageNet. For each dataset, we measure the percentage of utilized synapses in each layer as tasks are incrementally learned. Across all three datasets, we observe a consistent pattern: capacity usage increases rapidly during the initial tasks, then gradually slows
we analyze the synaptic capacity usage across three benchmarks: PMNIST,
19

---

Published as a conference paper at ICLR 2026
Table List of main notations
Notation
Description
Mj
Binary mask selecting active synapses for task j Accumulated mask of all previous tasks up to j Learnable relevance score for each synapse Layer-wise capacity ratio for subnet selection
M
r
P
Number of parallel synapses in each connection (synapse count)
wii
Weight of the p-th synapse from presynaptic neuron
N
Number presynaptic neurons
(t))
Membrane potential of spiking neuron at time
Tm
Membrane time constant in LIF neurons
Viest
Resting potential of the spiking neuron Total synaptic input current at time t Postsynaptic potential from synapse of neuron i
I(t)
(t)
To.áp
Decay constant of the p-th parallel synapse of neuron Synaptic kernel of the p-th synapse of neuron Arrival time the f-th spike at synapse (i.p) Eligibility trace representing local synaptic activity Decay time constant of the eligibility trace Modulation function for
(t)(t)
Smod (2)
Table 9: Integration with different types of methods.
Type
Method
10-split CIFAR-100
TinylmageNet
ACC (%
BWT (%) f
ACC (%) f
BWT (%) 1
EWC
72.77 (:: 0.57)
-3.59 (# 0.49)
64.51 (# 0.44)
-0.04 ( 0.03)
regularization-ban
EWC+MSCN
73.26 0.66)
-2.78 (# 0.19)
64.98 0.54)
-0.03 0.01)
ER
70.07
-7.70 0.59)
48.32
-19.86 0.70
replay-based
ER+MSCN
71.13 0.62)
-5.24 (# 0.51)
49.26
+ 0.84)
-9.17 0.55)
architccture-based
Bayesian
75.57 0.38 )
0.00 (11 0.00)
73.93
1: 0.36)
0.00 1: 0.00)
Bayesian+MSCN
76.48
0.00 0.00
74.56
0.00 + 0.00
down as more tasks are introduced. This effect is particularly pronounced in the fully connected layers, such FCI, which tend to accumulate more synaptic updates compared to early convolutional layers. The underlying reason is that the model needs to allocate new synaptic resources to encode novel task task-specific features at the beginning. However, training progresses, many tasks be handled by reusing synapses that represent similar features, reducing the need for additional capacity. This confirms the model's ability reuse past representations more effectively it acquires more knowledge, leading to slower growth in capacity usage over time.
A.3.3 ANALYSIS OF SYNAPSE COUNT
Fig. 9 iillustrates the relationship between synapse count (denoted as P) and average accuracy as the number of tasks increases, evaluated on PMNIST and SIFAR-100 Split. We vary the number of synapses per connection across five settings (P=2, 4, 6, 8, 10) and track model performance throughout the incremental learning process. We observe that on PMNIST, accuracy remains high across all configurations; however, larger synapse counts (e.g., P-8, 10) tend to deliver more stable performance over multiple tasks. On CIFAR-100 Split, the benefits increased synaptic capacity become more evident: higher P values consistently result in better average accuracy, particularly as the number of tasks grows. These results confirm that synaptic multiplicity enhances the model's ability to retain knowledge and generalize over longer task sequences.
20

---

Published as a conference paper at ICLR 2026
(a) PMNIST
(b) CIFAR-100 Split
(c) TinylmageNet
0.9
0.9
Camx2
Conua
Carna
CAP (%6
FC2
FEL
0.7
0.7
0.7
0.6
0.6
0.6
0.4
2
3
6
9
10
2
5 6
8 9
10
1
11
16
21
26
36
Task
Figure 8: Synaptic capacity usage as the number tasks increases on three benchmarks. Each curve shows the percentage of active synapses per layer tasks are incrementally introduced.
Accuracy Tasks (PMNIST)
Accuracy by Tasks (CIFAR-100 Split)
97.35
80.15
P=2
MCC (S)
97.22
76.75
96.88
73.35
97.45
80.55
Pu.4
ACC (96)
77.45
06.72
74.35
71.25
97.55
83.45
P-6
P
ACC (96)
79.18
96.95
74.92
97.55
80.05
Pw8
PeB
(9C.
98.68
73.92
97.35
02.25
Pw10
P-10
ACC. (9))
96.95
74.65
Increasing No. of Tasks
Increasing No. of Tasks
Figure 9: Average accuracy with increasing number of tasks under different synapse counts.
A.3.4 ORDER ROBUSTNESS ANALYSIS
To further investigate the robustness of our method to task permutations, we conduct additional experiments on the CIFAR- 100 Split benchmark using five randomly shuffled task orders. Fig. I0 presents the per-task accuracy across all 10 tasks for three representative baselines-EWC Kirkpatrick et al. (2017). GPM Saha et al. (2021). , and WSN Kang et al. (2022a) alongside our proposed MSCN. We observe that EWC (Fig. 10aa and GPM (Fig. 10b) are highly sensitive task order, exhibiting considerable variance in accuracy for the same task index across different permutations. contrast,
WSN (Fig. 10%) achieves more stable performance, though moderate fluctuations persist, particularly
21

---

Published as a conference paper at ICLR 2026
on later tasks. Notably, our method, MSCN (Fig. I0 maintains consistently high accuracy across all permutations and task indices, with significantly reduced inter-order variance. These results show that MSCN is robust to task order, ensuring stability in dynamic environments.
Performance Each Task
Performance at Each Task
70
70
OrdarA,
Orders
Oxtert
OrduD
Ordart
OrderA
Orderd
OrderC
OrdarD
Order
60
Accuracy (%)
(a)
Performance Each Task
Performance at Each Task
90
OrderA OrderB Onderi
OrderD
OrderE
OrdenA
Order
Oriern
OrderD
OrderE
80
Accuracy (96)
70
40
1
2
3
5
10
2
10
WSN
(d) MSCN
Figure 10: Task order robustness comparison on CIFAR-100 Split. Bar plots show per-task accuracy under five different task sequences.
22


Reviews:
[{"review_id": "yhTpEhlfU8", "reviewer": "Reviewer_jVm8", "summary": "This paper proposes a continuous learning framework for multi-synaptic cooperative networks (MSCN), inspired by the structural and plasticity mechanisms of multi-synaptic connections in biological neural systems. By introducing multi-synaptic connections within a fixed network architecture and integrating plasticity regulation based on local synaptic activity, MSCN aims to enhance the model's representational capacity, resistance to forgetting, and robustness to task sequence variations in continuous learning tasks. Experiments conducted across multiple benchmark datasets—including PMNIST, CIFAR-100, TinyImageNet, and 5-Datasets—covering both spiking neural networks (SNNs) and artificial neural networks (ANNs) demonstrate that MSCN outperforms existing state-of-the-art methods in terms of accuracy and forgetting suppression", "questions": "1. Can the author clearly specify the specific experimental settings for all comparison methods?\n\n2. The paper demonstrates MSCN's robustness to task orderings. How does the overlap degree of synaptic masks m_j vary under different task sequences? Furthermore, how does the qualification trace f_mod mechanism dynamically preserve or allocate synapses across different task orders?\n\n3. All current experiments are conducted under task-specific incremental learning settings. Could the authors conduct experiments in more realistic and challenging incremental learning scenarios?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. This paper introduces and models the multi-synaptic coordination mechanism in continuous learning for the first time. Unlike traditional network expansion or pruning methods, MSCN enhances the overall model's capacity by strengthening the representational power of individual connections.\n    \n2. The paper has a clear structure, with the Methods section describing the implementation of both SNN and ANN.\n    \n3. This work provides new insights for bio-inspired artificial intelligence models.", "weaknesses": "1. Table 1 compares MSCN with various baseline methods such as EWC, GPM, PackNet, SupSup, etc. However, the paper does not explicitly state whether all baseline methods were run under the identical “multi-head, task label known” setting.Regularization-based and replay-based methods are typically employed more frequently in “class-incremental learning” settings. Comparing their performance in this context to architecture-based approaches (PackNet, SupSup, WSN, MSCN) that achieve BWT=0 in “task-incremental learning” settings is profoundly unfair and exaggerates the latter's advantages.\n\n2. The core modulation function f_mod (Equation 7) in this paper is directly referenced from the work of Zhang et al. (2023). While its integration with a multi-synaptic architecture constitutes a contribution, the paper fails to clearly delineate in its presentation which elements are cited and which represent original innovations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RdoZDVpJVl", "reviewer": "Reviewer_CnT3", "summary": "This paper proposes the Multi-Synaptic Cooperative Network (MSCN), a biologically inspired framework for continual learning. The core idea is to model multiple synaptic connections between neuron pairs, modulated by local synaptic activity via eligibility traces, to enhance representational capacity and robustness within a fixed network architecture.", "questions": "1. How does the proposed multi-synaptic connectivity fundamentally differ from, for example, maintaining a per-parameter importance score (as in regularization-based methods) or learning a supermask over a fixed, over-parameterized network? What specific capabilities does the explicit modeling of multiple, independent synapsesenable that these other approaches do not?\n2. The eligibility trace mechanism is a key component. In the context of ANNs trained with backpropagation, what is the precise advantage of this bio-inspired, activity-dependent modulation over a standard gradient-based update? The ablation shows it helps, but the theoretical or empirical reasonwhy this specific form of modulation is particularly effective for task-order robustness remains unclear.\n3. The results show performance saturates with increasing P. Was there any exploration into making P adaptive or variable across layers/connections? The biological system it aims to emulate is highly heterogeneous, not uniform.\n4. Have you considered comparing against a strong, non-biologically-inspired baseline that also uses a fixed but over-parameterized network (e.g., a simple, wider network) to see if the gains are specifically due to the structured, multi-synapticnature of the over-parameterization, rather than just increased capacity?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The idea of integrating multi-synaptic cooperation with localized plasticity modulation is biologically inspired and potentially novel.\n2. The focus on improving robustness to task-order variation and scalability without dynamic network expansion targets significant practical limitations of existing architecture-based methods.", "weaknesses": "1. The abstract and introduction present the work as \"the first continual learning framework that explicitly leverages the multi-synaptic collaboration mechanism.\" This claim is overstated and needs significant qualification. The concept of having multiple, plastic pathways or \"synapses\" between units is not new. \n2. The authors must clearly differentiate MSCN from these related approaches. Is the novelty primarily in the specific biological inspiration(the explicit modeling of multiple contacts per axon-dendrite pair) and its integration with eligibility traces? This needs to be precisely stated and contrasted with the prior art.\n3. The statement that \"the brain achieves continual learning without suffering from dynamic structural growth\" is a simplification. While the overall neuron count is relatively stable in adulthood, synaptic formation and pruning (structural plasticity) are fundamental to learning.\n4. While capacity (CAP) is analyzed, the direct computational cost (FLOPs, memory footprint, training/inference time) of maintaining and updating P synapses per connection is not thoroughly discussed. For a method aiming for scalability, this is an important practical consideration. Table 3 mentions training time but under a fixed parameter budget; a discussion of the inherent cost of the multi-synaptic structure is warranted.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XpyOJqKcry", "reviewer": "Reviewer_dFvv", "summary": "This paper mainly focuses on the catastrophic forgetting in continual learning and presents a biologically inspired framework termed the Multi-Synaptic Cooperative Network (MSCN). Drawing inspiration from the organization of biological neural systems, MSCN enhances the representational capacity of a fixed network architecture by associating multiple plastic synapses with each neuronal connection, thereby enabling flexible information encoding without structural expansion. In addition, the authors propose a bio-inspired plasticity modulation mechanism that employs local eligibility traces to adaptively regulate synaptic updates. During learning, this mechanism selectively strengthens, weakens, or maintains specific synapses based on their relevance to the current task. This strategy helps prevent interference between tasks and reduces forgetting. Extensive experiments on multiple benchmark datasets demonstrate the superior performance and robustness of MSCN compared to existing approaches", "questions": "1.In Figure 3 (a–c), the current grayscale visualization makes it somewhat difficult to distinguish key patterns. Using color plots or more distinctive contrast could improve readability and help highlight the differences between conditions more clearly.\n2. Moreover, in Figure 3, EWC shows large accuracy fluctuations on the first task, likely due to task difficulty variations, whereas MSCN maintains notably stable performance. It would be helpful to clarify how MSCN achieves such robustness to task order and difficulty.\n3.Most baseline comparisons are made against architecture-based methods, yet the manuscript does not clarify the network size or total parameter count. Since each connection in MSCN contains K synapses, it is unclear how this differs in practice from simply increasing the network width by a factor of K. When K = 3, for instance, does the improvement primarily stem from the increased parameter count rather than the proposed mechanism itself?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.Sound methodology: The proposed MSCN framework is well-motivated by biological inspiration and conceptually well-grounded. The idea of equipping each neuronal connection with multiple synapses to enhance representational capacity within a fixed architecture is both intuitively appealing and theoretically sound.\n2.Comprehensive experiments: The experimental evaluation is thorough and well-executed. The paper benchmarks MSCN on multiple datasets using both spiking neural networks (SNNs) and artificial neural networks (ANNs), demonstrating its broad applicability across architectures. Moreover, the analysis under various experimental settings—such as different task order scenarios—provides a comprehensive understanding of the method’s robustness and generalization ability.", "weaknesses": "1. Lack of clarity: Although the overall framework is well-structured, the definition and notation of the multiple-synapse mechanism are somewhat ambiguous. In particular, the variables such as s, j, and f in Equations (2) and (3) are not clearly explained.\n2. Limited performance improvement and trade-offs: The performance gains of MSCN are relatively modest, particularly on the 5-Datasets benchmark. Moreover, as shown in Table 3, although MSCN achieves shorter training times, this advantage comes at the cost of reduced performance, suggesting a trade-off between efficiency and accuracy that warrants further discussion.\n3. Scalability and applicability to large-scale settings: The current experiments are limited to datasets with relatively small numbers of classes and models of modest size (e.g., MLPs, ResNet-18). The absence of evaluations on large-scale datasets such as ImageNet-1K or on modern architectures like Vision Transformers (ViT) makes it difficult to assess MSCN’s scalability and practicality in more complex, real-world scenarios. In particular, the computational and memory overhead, as well as the training stability of multi-synaptic connectivity in large models, remain unexplored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2uvUnbEuFU", "reviewer": "Reviewer_M3Db", "summary": "This paper proposes the Multi-Synaptic Cooperative Network (MSCN), a biologically inspired continual learning framework that models cooperative interactions among multiple synapses.The authors draw inspiration from multi-synaptic connectivity in biological neurons and local synaptic plasticity modulated by eligibility traces. Each neural connection is modeled as having multiple parallel synapses, whose plasticity is modulated locally based on activity. This enables adaptive task-dependent plasticity, allowing the model to selectively activate or inhibit synapses during new task learning while minimizing interference with prior knowledge. Extensive experiments across both spiking and non-spiking architectures on four benchmark datasets demonstrate that MSCN consistently outperforms state-of-the-art methods in both accuracy and robustness.", "questions": "My major concerns lie in the applicability of the proposed method. Please refer to the Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper is well-written and easy to follow.\n\n2. The proposed Multi-Synaptic Cooperative Network (MSCN) provides an innovative, biologically-grounded strategy for expanding network capacity in continual learning.\n\n3. The proposed method applies to both spiking and non-spiking neural networks, and achieves 0 BWT.", "weaknesses": "1. The experiments are mainly performed with CIFAR-100 and Tiny-ImageNet of relatively small image scales. Does the proposed method apply to larger-scale images, such as 224*224 images of ImageNet (subsets)?\n\n2. The proposed method seems to provide very marginal performance gains under more clear distribution shifts (i.e., 5-Dataset).\n\n3. As an architecture-based method, similar to other baselines, the proposed method may apply to only task-incremental setting.\n\n4. What’s the total parameter cost of the proposed method, compared to other baselines?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)