Decentralized Dynamic Cooperation of Personalized Models for Federated Continual Learning
Authors: Danni Yang
Zhikang Chen
Sen Cui
Mengyue Yang
Ding Li
Abudukelimu Wuerkaixi
Haoxuan Li
Jinke Ren
Mingming Gong
Venue: NeurIPS 2025 poster ()
Keywords: federated continual learning
decentralized dynamic cooperation
coalitional affinity game
Primary area: deep_learning

Abstract: Federated continual learning (FCL) has garnered increasing attention for its ability to support distributed computation in environments with evolving data distributions. However, the emergence of new tasks introduces both temporal and cross-client shifts, making catastrophic forgetting a critical challenge. Most existing works aggregate knowledge from clients into a global model, which may not enhance client performance since irrelevant knowledge could introduce interference, especially in heterogeneous scenarios. Additionally, directly applying decentralized approaches to FCL suffers from ineffective group formation caused by task changes. To address these challenges, we propose a decentralized dynamic cooperation framework for FCL, where clients establish dynamic cooperative learning coalitions to balance the acquisition of new knowledge and the retention of prior learning, thereby obtaining personalized models. To maximize model performance, each client engages in selective cooperation, dynamically allying with others who offer meaningful performance gains. This results in non-overlapping, variable coalitions at each stage of the task. Moreover, we use coalitional affinity game to simulate coalition relationships between clients. By assessing both client gradient coherence and model similarity, we quantify the client benefits derived from cooperation. We also propose a merge-blocking algorithm and a dynamic cooperative evolution algorithm to achieve cooperative and dynamic equilibrium. Comprehensive experiments demonstrate the superiority of our method compared to various baselines. Code is available at: https://github.com/ydn3229/DCFCL.

Decentralized Dynamic Cooperation of Personalized Models for Federated Continual Learning
Danni Yang', Zhikang Chen', Sen Cuil, Mengyue Yang', Ding Li', Abudukelimu Wuerkaixi', Haoxuan Li266 Jinke Ren", Mingming Gong
TTsinghua University "Peking University *University Bristol "The Chinese University of Hong Kong, Shenzhen 'The University of Melbourne Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) (hxli@stu pku. edu. en, mingming. gong@unimelb. Sanggminggggggguuiimellllll
edu au}
Abstract
Federated continual learning (FCL) has garnered increasing attention for ability to support distributed computation in environments with evolving data distributions. However, the emergence of new tasks introduces both temporal and cross-client shifts, making catastrophic forgetting critical challenge. Most existing works ag- gregate knowledge from clients into a global model, which may not enhance client performance since irrelevant knowledge could introduce interference, especially in heterogeneous scenarios. Additionally, directly applying decentralized approaches suffers from ineffective group formation caused task changes. To address these challenges, we propose a decentralized dynamic cooperation framework for FCL, where clients establish dynamic cooperative learning coalitions to balance the acquisition of new knowledge and the retention of prior learning, thereby ob- taining personalized models. To maximize model performance, each client engages in selective cooperation, dynamically allying with others who offer meaningful performance gains. This results -on-overl,,,,ng, variable coalitions each stage of the task. Moreover, we use coalitional affinity game to simulate coalition relationships between clients. By assessing both client gradient coherence and model similarity, we quantify the client benefits derived from cooperation. We also propose mmerge-blocking algorithm and dynamic cooperative evolution algorithm to achieve cooperative and dynamic equilibrium. Comprehensive experi- ments demonstrate the superiority of our method compared to various baselines.
Code is available at:
Introduction
Federated learning (FL), as distributed machine learning framework, addresses privacy and efficiency issues inherent in traditional centralized data processing I 2] Most existing works based fixed local data distribution aim to minimize a static joint objective. However, in real-world applications, clients continually collect new data over time, which leads to temporal catastrophic forgetting local sides, a critical challenge in continual learning (CL), which means parameters learned for past tasks drift toward new tasks during training.
To achieve FL in realistic scenarios with dynamic arrival of local data, federated continual learning (FCL) has been proposed. FCL faces two critical challenges: at local training stage, clients need to overcome temporal catastrophic forgetting induced by learning new tasks; at aggregation stage,
"qqual contribution. CCoresponning authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Before Aggregation
After Aggregation
Frd.ng
CTL
Per-kedAvg
DCFCL
0.8
Test Accuracy
0.6
Task ID
Task
0.4
9
5
2
3
10
Client ID
Client ID
Commenication Round
(0)
(b)
(c)
explain (a) and (b) show the effect of model aggregation on catastrophic forgetting. The circle size represents the
accuracy of each task on cach client, with larger circles indicating higher accuracy and smaller circles indicating lower. (e)
shows the impact of decentralized aggregation of personalized models on performance of federated continual learning.
Figure I: Spatial and temporal catastrophic forgetting in FCL.
spatial catastrophic forgetting should be addressed caused by knowledge interference from aggregated heterogeneous models. However, we assume aggregation can benefit clients in mitigating these issues, as learning from others facilitates acquisition of new knowledge and retention previous learning. To verify this conjecture, we trial our method on EMNIST [3] with 5 clients, each with 5 tasks in Fig. [a)(b), which show test accuracy of before and after aggregation. Before aggregation, models exhibit noticeable drift, heavily favoring new tasks. After that, accuracy on previous tasks improves significantly, underscoring influence of aggregation in alleviating catastrophic forgetting.
Although aggregation can mitigate catastrophic forgetting for personalized models, we believe the effect is uncertain, as clients may have incredible spatial data heterogeneity (4. Early studies adopt a central server architecture [5,6] 6 to aggregate, which performs poorly when facing strong heterogeneity. In fact, several decentralized methods have been developed in personalized FL 77. In Fig. c). we set up heterogeneous scenario on MNIST [9] to illustrate personalized (Per- FedAvg) [0] and decentralized aggregation (CFL) 77) significantly improve performance compared to centralized method (FedAvg) [11. By group aggregation in decentralization topology, it can promote effectiveness of aggregation and alleviate heterogeneous interference, therefore further mitigates catastrophic forgetting. However, directly transferring decentralization from FL to FCL suffers from ineffective grouping aggregation caused by task changing.
Inspired by above discussion, we introduce a novel decentralized Dynamic Cooperative Federated Continual Learning (DCFCL) framework to achieve personalized FCL., allowing clients to form non-overlapping coalition topology in each aggregation phase to prevent grouping ineffectiveness. These coalitions are composed by several subsets of clients who assist one another improving their respective model performance to facilitate personalized learning. We aim to identify coalitions to achieve cooperative equilibrium state, where no alternative coalitions would yield greater benefits for all cooperators inside. Equilibrium is dynamic, capable of disintegration or reorganization as tasks change, eventually leading to new equilibrium.
To achieve above framework, we utilize knowledge distillation to maintain model consistence to identify cooperators, then quantify and calculate client benefits in various coalitions based overall similarity-come compiiii gradient coherence and model similarity and coalitional affinity game to further formulate benefit table. After obtaining benefit table, we propose a merge-blocking algorithm to achieve cquilibrium state and dynamic cooperative evolution algorithm to evolve new equilibrium each aggregation phase. Through dynamic cooperative equilibrium, clients achieve personalized models in decentralized FCL framework. The main contributions of this paper are as follows:
We propose novel decentralized framework for personalized FCL. allowing dynamic coop- eration among clients to mitigate catastrophic forgetting and improve model performance.
We use overall similarity and coalitional affinity game effectively quantify and calculate client benefits in cooperative coalitions.
We propose merge-blocking algorithm to recognize cooperative equilibrium and dynamic cooperative evolution algorithm to quickly evolve new equilibrium at each aggregation.

---

Task
Task
Task 3
Client
Client 2
Client 3
Client 4
coalition
coalition
coalition
explain: In tusk I, clients 1, 2, and 3 have similar data distribution, so they cooperute, whereas 4's task differs from them, providing no mutual benefit. Thus, trains locally. task 2, clients and have similar distributions, leading to cooperate. Meanwhile, 2 cooperates with 3 to recall task The same is true task 3.
Figure 2: System model. Illustrate dynamic cooperation in decentralized federated continual learning.
2
Related Works
Continual Learning CL addresses common scenario in which tasks arrive continuous data stream for network to learn. Strategies like eggularization-based rehearsal-basede and dynamic architecture-based approaches are employed mitigate catastrophic forgetting. Regularization-basee methods like EWC [12] constrain changes in weights of previous tasks, thereby reducing catastrophic forgetting. Rehearsal-based approaches involve preserving data of previous tasks or generating pseudo-data [13) to train next task, like LUCIR [4] and iCaRL [15). Dynamic architecture- based methods encompass expanding models or employing parameter isolation to retain previous knowledge. such as Piggyback [66], WSN [17], and Lwl (18)
Federated Learning FL is typically categorized into centralized and decentralized frameworks. Centralized FL [9] like FedAvg [11, FedProx [20]. and SCAFFOLD 21 involve aggregating locally trained models from individual clients central server to obtain global model. Decentralized FL is tailored for client needs. Hypernetworks are introduced to enable decentralized cooperative FL
22
23
Decentralized protocol is also proposed support personalized learning 24 10
Federated Continual Learning FCL considers not only catastrophic forgetting but also irrelevant knowledge interference. Knowledge distillation is used for knowledge preservation 25 26 27
Replay is also extended from CL to FCL, like FedCIL (28) and AF-FCL [29]. These methods adopting centralization may lead suboptimal performance once substantial heterogeneity arises.
Cooperative Game Theory Cooperative game theory investigates strategy where players can achieve agreements on coalitions and benefits of cooperators 31 32]. Collaborating in FL is proposed to develop personalized models [23]. Cooperative game is also explored in resolving linear regression and mean estimation problems in FL 33 34 These works rely on static cooperative strategy formulating fixed coalitions, which may lose effectiveness due task variations. So we emphasize dynamic cooperative strategy for FCI.
Decentralized Federated Continual Learning
3.1 Problem Setup
In decentralized FCL architecture, there K clients forming the set K. (I,..., K} without a central server. Each client has local dataset D3 = {D],, , D.), where T denotes the ni total number of task phases and Dí. { vk Jim1 is the training data in phase containing ni samples and {at (i) the i-th data sample. y E C₁,, and de the class set of Dj. In practical scenarios, it may be observed that the task set of clients is not necessarily correlated. Thus we consider a practical setting, the limitless task pool (LTP),, denoted as T. For each client, datasct DJ, of the k-th client at time t corresponds to particular learning task T C T. There

---

is no guaranteed relation among the tasks (Tx Th..... ......T'. Similarly, time t, there could be no relation among the tasks (TI, ...... Tk} across different clients, i..., (T n |T 2 0, p 1.2... K. More importantly. clients possess diverse joint distributions of data and labels due to heterogeneity. Therefore, at aggregation phase, local models always deviate from their current tasks. Our goal is for decentralized FCL to enable clients acquire new knowledge while retaining prior learning through aggregation. Consequently, at each task phase i, model parameter of client is 0g and optimization goal of each client is:
Tx in the k-th client at different steps.
argmin (0 Th TE, TE). %
(1)
where L3 the risk objective of client k.
3.2 System Model
In decentralized FCL system, dynamic cooperation with others is good method to enhance the model's performance on current tasks while mitigating catastrophic forgetting of previous tasks. This scenario is illustrated in Fig. 2 Suppose there four clients, each with three tasks. Because of heterogeneity, the best model for a particular client is likely to come from cooperating with a subset of clients rather than all. At each task stage, clients select different cooperative partners based on the trade-off between acquisition of new knowledge and retention prior learning. The final cooperation result is equilibrium state composed of non-overlapping coalitions where all clients relatively satisfied with their current coalitions and do not shift to other groups With the constant arriving of new tasks, the equilibrium state for each task phase will evolve dynamically.
Assuming each client has T' tasks, during the round of local updates for task t. When the coalition structure that client & belongs to is S, the aggregated model 0% of client k, can be updated by the following steps:
(a) local iterations:
0%
+ 0% - 1)
(2)
followed by aggregation step that updates local model 0% 05+
with a combination of model updates
A 6% 0++ T--
(b) aggregation:
0-- = anO + + Σ a.0.
ax (0 + A0R) +
E ag(07 + +00)) = Σ Cn,00 + A07) (ES
iEST(A)
EE9
where og can be explained as weight coefficient of client i. Therefore, the optimization variable of determined by steps (a)(b) simultaneously, which can be subdivided into 1 S.
3.3 Cooperative Game
To achieve the optimization goal shown in I in the above-mentioneel system model, we introduce the concept of cooperative game, which is usually modeled as a process coalition formation 35 Using language of cooperative game theory, we can interpret cooperative state 8 round t as partition x consisted of non-overlapping coalitions between clients, as well as benefit vec- tor u(x) for each client, i.e., st,
(u(TT),)) m). There are Bx states for K clients forming a set For any state s (x(T) denotes benefit to k under corresponding parti-
sB S = [sT ..T }. tion и. We aim find an optimal state that yields 0% minimizing loss while maximizing ben- efit (i.e. (p(sTh)
max un uR(s),) maxg(sem(<7)) T. (iis 0;07; Diat) maxo: - (0%; Drat) mino; TAx (0%: De where - (8 optimization problem of becomes problem of cooperative game after local iteration and optimiza- tion variables include local model parameter 07 and coalition structure S. The coalition set is including all coalitions for clients. Based on different coalitions, clients can obtain various benefits. These coalitions and benefits can eventually formulate a benefit table.
-Lx(0,, Drat)), which can be achieved by: un (x])
A6 = Σies O, SE (T( is coalition that client & belongs to. The
{S1, ...,, S2x

---

Benefit
Trumition Process
State
State
Partition
State
State
Ca
2
2
3
4i
(GJ]HGghGG
<t
$j
sq
4
1 2 o [Cps (Cg)
<
8/
Cap, [Cy)
84
47
(Gy) (G2x Gj) 0 4 2
4
4
4
4
[C Cz.
(a)
(b)
(c)
explain: (a) benefit table; (b) is transition process of equilibrium formation, Arrows indicate transitions from previous one to next. Each state is different color und eventually reaches equilibrium sf: (c) next state at each transition corresponding to (b); (d) shows partition changes, and red dotted line represents blocking coalition which contributes to transition from previous to next.
Figure 3: Benefit table and state transition process with three clients as an example.
Achieving equilibrium for stable cooperation Fig. 3 a) shows an example of a benefit table with 3 clients, including 5 cooperative states, partitions and 7 coalitions. Obviously, there no coalition partition that allows all clients to reach their optimal benefit simultaneously. However, given the limited state space of coalition partitions, there is at least one equilibrium state where all clients are relatively satisfied with benefit in current coalition and will not deviate to other groups. To achieve the equilibrium state, we propose the concept of the transition process of equilibrium formation (TPEF), which involves transitioning from one state to another, ultimately reaching equilibrium. Transitions are driven by clients who can derive better benefits from forming coalition, known as profitable transition (PT). Assuming a state S and coalition S, then S has a weak PT from s if there is a state s, with S (s,)) such that Ug (s ) > Ux(ss,, for all & S, which means some clients can obtain the same or more benefits by forming coalitions with each other. When 2 turns into >, all clients can get more benefits than now, changing to a strict PT. Here S is called blocking coalition (BC). If there is strict PT, state must transfer. Once there is a client in S suffer from benefit loss, state doesn't change. s is equilibrium state if there no coalition state s with a blocking coalition S such that Vk E S, if k E S, < i < m then ( ) 2 Up (x and 3/ E 8, if / E Sj, 1 S S. m, then uy (x7)), )= > uy (s)))) As shown Fig. (b)) transition process is listed. At sf the coalition {C1, (BC) leads to better benefits for each, thus C1, C2 will cooperate, state transfers to sy. At 8j, C3 will betray {C1, C3} and switch to {C2, CB)(C), and state will transfer to $2. Any state will eventually transfer to sJ, which has no BC for it and thus represents equilibrium.
Dynamic Cooperative Strategy
Our goal to develop a dynamic cooperative strategy that achieves equilibrium at each aggregation stage. To accomplish this, we need to complete two key tasks: (1) Formulating benefit table. The most intuitive method involves creating various aggregation models based on different coalitions. These aggregation models are then used to test performance of all tasks local clients, which can determine client benefits. Theoretically, there are Bи cooperative states for K clients, where Bx is Bell number representing the number of ways to partition set with K elements. Given that exhaustively trying all aggregation models locally has extremely heavy computation and communication cost, we propose concept of overall similarity among clients to quantify 2-client benefits. Then, we coalitional affinity game to quickly calculate multi-client benefits. (2) Achieving dynamic cooperative equilibrium. Based on analysis of TPEF in 3.3 traversing TPEF of all states find equilibrium, however it requires exponential time complexity, so we explore efficient merge-blocking algorithm to achieve equilibrium and dynamic cooperative evolution algorithm to quickly evolve new equilibrium.
4.1 Preparatory Condition
Knowledge distillation for maintaining consistent features to identify cooperator When client trains on new task, the classifier is continuously modified by new features, which is not conducive to identifying cooperators who can assist in recalling previous knowledge. Therefore, maintain the consistency of the classifier's 's feature space to maximize the utilization of their own model information rather than extra information exchanging to identify cooperators efficiently.
We apply knowledge distillation in classifier control classifier's feature space preventing from drift to new task. First, there is one teacher model (past model of round 7 1) and one student model
5

---

((L2) 39)
1
cosigy 9t)
+e- cocLo, 92)
C2
C
(te S0looo
T(2.5)
Figure An affinity graph for 3-client coalition.
(current round r). Output logits for teacher model are denoted as (z) = [oī (z)...., where z is input network and is the dimension of logits vector, and logits of student model are o" (x: = [o (x:), ...,,....... 0% (x)] The distillation loss for client h on round r defined as:
Li
=
Σ xE
pf ( log [p] (x:),,
i=1
where 0%. is student model, and p. (x)
Σj=1 e (oy are temperature-scales logits, where F is
temperature scaling parameter. p. refer to predictions of teacher model (o" (o"=1(x)) and pt (x) refer student model (o" (z))). The classification loss in is
exp(o (I))
Lasss (0):
Σ (..N)
ys log
(5)
Σj=1 exp(o)
The final loss can be formulated as
D = Liaass + AL
(6)
where À scalar which regularizes influence of Liis
4.2 Formulating Benefit Table
In order to form a complete benefit table, we first propose concept of overall similarity to quantify benefits of 2-client coalition. Taking this backbone, calculate benefits of multi-client coalition based on theory of coalitional affinity game.
Benefit quantification with overall similarity To reduce communication and computing overhead, we utilize the model information rather than extra information exchanging to quantify client benefits. It is highlighted that finding descending direction close the local gradient for aggregating models can reduce conflicts caused by client heterogeneity [36 37 Inspired by this, we first quantify benefits through local model gradient coherence. However, relying solely on gradient coherence may aggregate heterogeneous models generating clients interference. This is because the model parameters of different clients may differ significantly overall, even if their gradients are similar. Therefore. we propose to incorporate global model similarity, as it contains essential global information. We comprehensively utilize these two similarity measures an overall similarity, considering both the coherence of gradient direction and the proximity of model parameters. For ease of representation, communication round т, we use gs, 9, to represent the gradient of client and j. and 6, and 0 to represent the model parameters. We use cosine similarity to calculate. Therefore, benefits under 2-client coalition can be defined as overall similarity of i and j, i.e.,
d4 = uj = cos(g,, 93) E cos cos(OO,,0 0,)
gt. 9j
+e*
00,0,
ag + be big
(7)
|1g
[
||0

---

where at and by represent gradient cosine similarity and model cosine similarity of i and j. respec- tively. e hyperparameter, when it equals to O, only gradient similarity represents benefits.
Benefit calculation with coalitional affinity game With the benefits of 2-client, we need to calculate benefits of multi-client coalition. Coalitional affinity game solution because it can model relationships between clients. It is kind of hedonic game that explicitly models the value that an agent receives from being cooperated with other agents [38]. We can use infer benefits in the multi-client coalition through the relationship between two clients. For any pair of clients, denote affinity of for j as r(i.j) E which represents benefit that receives from cooperating with and it is already quantified as overall similarity. We represent the clients and their affinities with
an affinity graph = {N, R). it is a weighted directed graph where edge r(2, j)) R represents an affinity relation between i and j. Taking 3-client coalition
example in Fig. 4 benefits for
2-client are weights on edges in affinity graph. According to affinity graph, benefit in multi-client
coalition can be defined as the function f(-) of benefit in 2-client coalition, i.e.,
O.
if S (i) if S [1,33
U, =
r(i,j),
(8)
f(r(i,11)) (i,,nn),,
if S {i,J1. In
Next, we prove the specific format of f() in Appendix Theoretically, benefit of is:
0y cos (Mavgs 9i) + cos (Oang: 0,) Σ Epest 9p 0paip119p11
<Epes apbipl 0pll
Σ EpeSrr0 0/1 + H
(9)
where
==
Σ
2apag9p94 =
Σ
2ppa
|l 11 III 9q
p. Σ
H
Σ
2apa,090
2apagbpy
02 0g
(10)
and E S\ {i} represents all clients in S except i. Davg is aggregated model of coalition. To sum up, define the benefit of i who belongs to coalition S as
if S (i),
04 = cos (gi, 97) + cos (0,, 0,) cos (guvg. 9:) + cos (Gaug. 0,)
if S {i.j}.
(11)
if S = [i.ji.-.. 5n}
We use the form of the weighted average of samples for model aggregation, where
1
Σ 9p np
gavg Eper)(0) Ti
pes
(12)
0 Davg EpER)(1) Pp peS)(()
Σ
0 Rpp
where 0p p ny Rpp represents sample number of p. At task t, it equals to ni. According to 9 benefit i in multi-client coalition can be represented by benefit in 2-client coalition. On account of this, we can formulate benefit table quickly by 2-client relationship.
4.3 Dynamic Cooperative Equilibrium
traversing all states is method to achieve equilibrium. However, is computationally intensive, with time complexity of O((Bx)² K). In (3)], merge-split algorithm used for coalition formation, but it only identifies local optimal solutions in the Pareto Order. Rational clients can benefit more by blocking coalitions in PFCL, therefor equilibrium is ultimately
Based on analysis of TPEF 3.3
7

---

Algorithm 1: Merge-Blocking Algorithm
Input: The initial partition Fin
Output: The final partition n²
Sort coalitions set & in ascending order by the number of clients of each coalition; Set Tup f- Tin, pree f- 0, Count Table CT + 0, Stable Coalition SC' <- 0, π t 0;
while Tap + Tprev and Tap + do
Set Tyrev f- Tupi
Set CT +;;
for S. E do
Tup [Si,.... 12), Nnew {S upp)
Tup is the new set after coalitions in it has removed the elements contained in S;
if (all((;(Tnew)) 2 uz(Fup)li E 8) and (u;y(u;(Tann) > (z(Fup)li E S*) then Set Tup f Tnewi
Remove all S; E CT with Si & Tup and add counts in CT of S E Tapi
if (CT(CT) + then Set SC mar mar(CT); Tup Tup SSC;
for S E S do if set set (S)))((( then S + S;
x USC;
else
n t x U uupi
stable result. Motivated by this, we develop merge-blockin algorithm to achieve cooperative equilibrium and iteratively evolve new equilibrium through dynamic cooperative evolution algorithm.
In Algorithm|] [ traverse coalitions, which have less quantity than states, reduce computation. We begin with singletons for each client as initial partition and iteratively traverse coalition set When current partition encounters BC: S S, clients in partition are merged forming S and previous coalitions are blocked and reorganized. To reduce traversals, we introduce stable coalition (SC) prune. By tracking the frequency of coalitions in update partition, we identify SC' with maximum counts accumulated and cannot be blocked by any other BCs. Then we remove all coalitions from which contain clients of SC and then continue traverse S find the next SC until there is no BC. Ultimately, equilibrium partition is the collocation of all SCs. Our simulation results indicate that Algorithm [] converges to equilibrium within only few traversals. After achieving equilibrium, we evolve new equilibrium by dynamic cooperative evolution algorithm to realize dynamic equilibrium among clients at each aggregation stage. See Appendix for more details.
Experiments
5.1 Experimental Settings
Datasets and baselines. We conduct 4 datasets on different settings. 1) EMNIST-LTP 3:: character classification dataset with classes. 2) EMNIST-shuffle [3): the task sets of EMNIST are arranged in different orders. 3) CIFARI00 [40]: a challenging image classification datase. 4) MNIST-SVHN-F
42 The dataset is constructed with MNIST (.. SVHN (41 and
FaShionMNIST 42 We compare our method with 5 FL. baselines, 2 CL baselines and 6 FCL. baselines. See Appendix for more details dataset settings and baselines.
5.2 Experimental Results All Datasets
In EMNIST-LTP dataset, clients may encompass unrelated tasks, thus rendering the dataset challeng- ing. The performance of all methods on EMNIST-LTP shown in Table Our approach exhibits superior performance across all of the comparative experiments. Different from EMNIST-LTP, EMNISS-shuffle represents more tractable dataset within the conventional setting, resulting in higher overall accuracy rates as in Table [ Our method still showcases superior capacity than all

---

Table 1: Average accuracy on all datasets.
Model
EMNIST-LIP
EMNIST-shufit
CIFAR100
MNIST-SVHN-F
FedAvg
20.9
70.3
26.3
55.7
FedProx
35.3
69.4
28.7+1.4
56.1
SCAFFOLD
30.1
74.7
37.4+1.2
41.640.9
CFL
44.5
71.640.3
35.1
59.2
Per-FedAAg
46.2
75.2
35.9
54.1
PODNet+FedAvg
36.9 1.3
71.0
30.5
54.2
PODNet=FedProx
40.4-0.4
70.6
32.5
56.4+0.4
ACGAN+FodAvg
38.4=0.2
70.0
32.1+1.6
56.0+0.7
ACGAN+FedProo
41.3-0.9
70.3+1.2
31.8+0.7
56,4-2.1
FLwF2T
40.1
71.0=0.9
30.2+0.7
54.2+0.6
FedCIL
42.0
71.1
33.5
57.2+1.7
GLFC
40.1
14.9
35.640.6
61.8
AF-FCL
40.5
75.8
36.3
68.1
AFCL
40.7
27.0
32.3
62.4.0.6
FPPL. 41.4 40.6
76. 10.9
31.5
61.7
DCFCL
52.5
78.3
40.4
66.7+0.9
baselines in this commonly adopted dataset setting. In addition, as data heterogeneity becomes more severe (from EMNIST-shuffle EMNIST-LTP), our method achieves greater performance compared to others. This is likely because increased data heterogeneity leads to substantial variations among models. Consequently, aggregating knowledge from clients into a global model potentially result in conflicting knowledge. In such scenarios, our decentralized federated learning is more effective.
Table also displays the results of two more challenging datasets: CIFARI00 and MNIST-SVHN-F. By aggregating highly correlated models, our method guarantees client benefits in terms of both optimization direction and global consistency, significantly exceeding performance of most baselines.
5.3 Ablation Studies
Our method consists of three main components: (i) Cooperative Equilibrium (CE). We introduce Dynamic Cooperation in decentralized FCL. Global cooperation transfers to FedAvg, and non- cooperation degenerates into Local algorithm, where clients execute the CL process locally without any aggregation. (ii) Knowledge Distillation (KD). We use knowledge distillation loss to maintain consistent features classifier during training to identity cooperator, as can prevent feature drifts. (iii) Overall Similarity (OS). To quantify client benefits, we propose overall similarity. When approaches 0, it degrades to only use gradient coherence for quantification.
Table Ablation studies on EMNIST-LTP and EMNIST-shufIle datasets.
Model w/o CE-FedAvg
EMNIST-LIP
EMNIST-shufil
32.5
70.3
w/o CE-Local
12.3
17.3
w/o KD
50.340.3
73.2+0.4
w/o OS
45.340.8
73.7
DCFCL
52.5
78.3
We conduct ablation studies on EMNIST-LTP and EMNIST-shuffl datasets as displayed in Table 2 Our method achieves optimal performance with all three modules. The accuracy of Local is incredibly low, which reflects the significance of decentralized cooperation for FCL.
5.4 Results for Different Parameter Settings
We conduct experiments on EMNIST-LTP and EMNIST-shufite datasets with various À and E. is fixed at 0.2 when is varied, and vice versa. As shown in Table emphasizing model similarity by increasing enables clients to identify peers with more aligned feature spaces for learning. Therefore, is essential to determine the optimal overall similarity composition. addition, we also adjust to illustrate the influence of knowledge distillation. Increasing À retains more prior task information

---

for cooperator identification, which in turn promotes more effective cooperation and alleviates catastrophic forgetting.
Table Average accuracy on EMNIST-LTP and EMNIST-shuffle datasets with variable parameters.
Parameter
EMNIST-LTP
EMNIST-shuffl
Parameter
EMNIST-LTP
EMNIST-shuffle
0.0
44.340.8
75.940.3
0.0
50.3
73.2
0.2
40.5
78.340.6
0.2
52.5
78.3
0.4
48.7
80.2
0.4
50.7
78.7
0.6
45.1+1.0
70.4
0.6
51.3+0.7
74.0
0.8
46.5
71.440.6
0.8
53.7
77.7
1.0
40.5
72.240.8
1.0
55.7
81.3
Conclusion
This study pays attention to critical challenges of temporal and spatial catastrophic forgetting in federated continual learning. We propose decentralized dynamic cooperative learning framework that personalizes client models. Clients form non-overlapping dynamic coalitions at each aggregation stage to mitigate catastrophic forgetting and further improve performance. The experimental results clearly demonstrate its effectiveness. Whereas, some parameter sensitivity remains(e.g. X, E), which could affect performance in unseen settings. Exploring adaptive mechanisms is left for future work.
Acknowledgments and Disclosure of Funding
MG supported by ARC DP240102088 and WIS-MBZUAL 142571. Sen Cui would like to acknowledge the financial support received from Shuimu Tsinghua scholar program. HL was supported by National Natural Science Foundation of China (623B2002). JR supported by the National Natural Science Foundation of China under Grant No. 62501514.
References
[1] Hao Li, Chengcheng Li, Jian Wang, Aimin Yang, Zezhong Ma, Zunqian Zhang, and Dianbo Hua. Review on security of federated learning and its application in healthcare. Future Generation Computer Systems, 144:271-200 2023.
[2] Xiaokang Zhou, Xiaozhou Ye, Kevin I-Kai Wang, Wei Liang, Nirmal Kumar C. Nair, Shohei Shimizu, Zheng Yan, and Qun Jin. Hierarchical federated learning with social context clustering- based participant selection for internet of medical things applications. IEEE Transactions on Computational Social Systems, 10(4):1742-1751, 2023.
[3] Gregory Cohen, Saeed Afshar, Jonathan Tapson, and Andre Van Schaik. Emnist: Extending mnist to handwritten letters. In 2017 International Joint Conference on Neural Networks (IJCNN), pages 2921-2926. IEEE, 2017.
[4] Xin Yang, Hao Yu, Xin Gao, Hao Wang. Junbo Zhang, and Tianrui Li. Federated continual learn- ing via knowledge fusion: A survey. IEEE Transactions Knowledge and Data Engineering, 36(8):3832-3850, 2024.
[5] Jachong Yoon, Wonyong Jeong. Giwoong Lee, Eunho Yang, and Sung Ju Hwang. Federated continual learning with weighted inter-client transfer. In International Conference on Machine Learning, pages 12073-12086. PMLR, 2021.
[6] Donald Shenaj, Marco Toldo, Alberto Rigon, and Pietro Zanuttigh. Asynchronous federated continual learning. In 2023 IEEE/CVF Conference Computer Vision and Pattern Recognition Workshops (CVPRW), pages 5055-5063, 2023.
[7] Felix Sattler, Klaus-Robert Müller, and Wojciech Samek. Clustered federated learning: Model- agnostic distributed multitask optimization under privacy constraints. IEFE Transactions on Neural Networks and Learning Systems, 32(8):3710-3722 2021.
10

---

[8] Leijie Wu, Song Guo, Yaohong Ding, Yufeng Zhan, and Jie Zhang. A coalition formation game approach for personalized federated learning, 2022.
[9] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11))227-2) 23224, 1998.
[10] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning: A meta-learning approach. 02 2020.
[11] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-eefieee efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics, pages 1273-1282. PMLR, 2017.
[12] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaune Desjardins. Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Baabbka-Baarwiska. et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of 2017.
sciences, 1 114(13):3521-33)
[13] Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. Continual learning with deep generative replay. Advances in neural information processing systems, 30, 2017.
[14] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 831-839, 2019.
[15] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceddiggs of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001-2010, 2017.
[16] Arun Mallya, Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting single network to multiple tasks by learning to mask weights. In Proceedings of the European conference on computer vision (ECCV), pages 67-82. 82, 2018
[17] Hacyong Kang, Rusty John Lloyd Mina, Sultan Rizky Hikmawan Madjid, Jachong Yoon, Mark Hasegawa-Johnson, Sung Ju Hwang, and Chang D Yoo. Forget-free continual learning with winning subnetworks. In International Conference on Machine Learning, pages 10734-10750. PMLR, 2022.
[18] Zhikang Chen, Abudukelimu Wuerkaixi, Sen Cui, Haoxuan Li, Ding Li, Jingfeng Zhang, Bo Han, Gang Niu, Houfang Liu, Yi Yang, et al. Learning without isolation: Pathway protection for continual learning. arXiv preprint arXiv:2505. 18568, 2025.
[19] Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. Federated learning with matched averaging. arXiv preprint arXiv:2002.0644 2020.
[20] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429 450, 2020.
[21] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. SCAFFOLD: Stochastic controlled averaging for federated learning. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 5132-5143. PMLR, 13-18 Jul 2020.
[22] Aviv Shamsian, Aviv Navon, Ethan Fetaya, and Gal Chechik. Personalized federated learning using hypernetworks. In International Conference Machine Learning, pages 9489-95022 PMLR, 2021.
[23] Sen Cui, Jian Liang, Weishen Pan, Kun Chen, Changshui Zhang, and Fei Wang. Collaboration equilibrium in federated learning. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 241-251, 2022.

---

[24] Rong Dai, Li Shen, Fengxiang He. Xinmei Tian, and Dacheng Tao. Dispfi: Towards communication-efien personalized federated learning via decentralized sparse training. In International conference machine learning, pages 4587-4604, PMLR, 2022.
[25] Yuhang Ma, Zhongle Xie, Jue Wang, Ke Chen, and Lidan Shou. Continual federated learning based on knowledge distillation. In IJCAI, pages 2 2182-2188, 2022.
[26] Anastasiia Usmanova, François Portet, Philippe Lalanda, and Germán Vega. A distillation-based approach integrating continual learning and federated learning for pervasive services. CoRR, htss2109.00197, 2021.
[27] Jiahua Dong, Lixu Wang. Zhen Fang. Gan Sun, Shichao Xu, Xiao Wang, and Qi Zhu. Fed- erated class incremental learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA. USA, June 18-24, 2022, pages 10154-10163, 2022.
[28] Daiqing Qi, Handong Zhao, and Sheng Li. Better generative replay for continual federated learning. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023.
[29] Abudukelimu Wuerkaixi, Sen Cui, Jingfeng Zhang, Kunda Yan, Bo Han, Gang Niu, Lei Fang, Changshui Zhang, and Masashi Sugiyama. Accurate forgetting for heterogeneous federated continual learning. In The Twelfth International Conference on Learning Representations, 2024.
[30] Esther M. Arkin, Sang Won Bae, Alon Efrat, Kazuya Okamoto, Joseph S.B. Mitchell, and Valentin Polishchuk. Geometric stable roommates. Information Processing Letters, 109 224, 2009.
[31] R. J. Aumann and J. H. Dreze. Cooperative games with coalition structures. lnt. J. Game Theory, ((4):217-237) dec 1974.
[32] Sang-Seung Yi. Stable coalition structures with externalities. Games and Economic Behavior, 20(2):201-233 1997.
[33] Kate Donahue and Jon Kleinberg. Model-sharing games: Analyzing federated learning der voluntary participation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(6):5303-531, May 2021.
[34] Kate Donahue and Jon Kleinberg. Optimality and stability in federated learning: a game- theoretic approach. In Proceedings of the 35th International Conference Neural Information Processing Systems, NIPS '21, Red Hook, NY, USA, 2024, Curran Associates Inc
[35] Hideo Konishi and Debraj Ray. Coalition formation as a dynamic process. Journal of Economic Theorr, 110(1):1-41, 2003.
[36] Zexi Li, Tao Lin, Xinyi Shang, and Chao Wu. Revisiting weighted aggregation in federated learning with neural networks. In Proceedings of the 40th International Conference on Machine Learning, ICML/23. JMLR.org, 2023.
[37] Yasaman Esfandiari, Sin Yong Tan, Zhanhong Jiang, Aditya Balu, Ethan Herron, Chinmay Hegde, and Soumik Sarkar. Cross-gradient aggregation for decentralized learning from non- iid data. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 3036-3046. PMLR, 18-24 Jul 2021.
[38] Simina Brânzei and Kate Larson. Coalitional affinity games and the stability gap. IJCAI'09. page 79-84, San Francisco, CA, USA, 2009. Morgan Kaufmann Publishers Inc.
[39] Ning Zhang, Qian Ma, Wuxing Mao, and Xu Chen. Coalitional fl: Coalition formation and selection in federated learning with heterogeneous data. IEEE Transactions on Mobile Computing, 23(1 1041144--400000 2024.
[40] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers 2009.
of features from tiny images.
12

---

[41] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. 2011.
[42] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. CoRR, abs/1708.077477 2017.
[43] Moming Duan, Duo Liu, Ji Xinyuan, Renping Liu, Liang Liang, Xianzhang Chen, and Yujuan Tan. Fedgroup: Efficient federated learning via decomposed similarity-based clustering. pages 237, 09 2021.
228-237,
[44] Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distillation for small-tasks incremental learning. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XX 16, pages 102. Springer, 2020.
[45] Chenshen Wu, Luis Herranz, Xialei Liu, Yaxing Wang, Joost van de Weijer, and Bogdan Raducanu. Memory replay gans: Learning to generate new categories without forgetting. In Advances in Neural Information Processing Systems 31: Annual Conference Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 5966-5976, 2018.
[46] He and et al. Fppl: An efficient and non-iid robust federated continual learning framework. 2024.
[47] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 770-778, IEEE Computer Society, 2016.
13

---

A Proof of Theorem
We have obtained the client benefit r(t,j) under overall similarity. If a coalition structure has client models z, y, 2, then we have
r(a, 2) Oxz + E * bas
rly =) ays + bys
(13)
r(r.y) y) = azy + * Day
where auy and bay present gradient coherence and model similarity of x and y, respectively. auz and baz present gradient coherence and model similarity of x and z, respectively, ayz and byz present gradient coherence and model similarity of and z, respectively.
Then the client benefit of z in S can defined as the overall similarity of the model to z after a and y are aggregated. The gradient and model after aggregation respectively
Bang = 02 + 0gBy: Javg 0z + 0y99v
(14)
where 02 and Og can be explained as the aggregation weight of client a and y. For the aggregation model, we have
oong 02 = a20002 + t,,000
900g92 = angale + 0y9y92
00ag
a |0,1 12 + a2110,11 2| 2x 0.00
(15)
gavgl
o || 12 a by 112 2k
Then the client benefit of can be expressed as
uz = Casgg 9z) + E cos(faeg 02)
ar929: + 0y9992
a,000= + 8,8y0
119avel| ||
1vull1 |0111
0g = 1- ag y2 l
(16)
V a [02112211
a 1,y 11 + 20 Ct Oxy 1l
1l
a bus |0
+ ayb 16
+ +
02 2| a |1,
2a a bay 1z |10,11
where
9=9= axz
9x
||g=1
0_0_ buz
|0_
||0_1]
(17)
9y9: ayz
|1y
[|9=1]
8,0= bys |0,
1|8=11
Similarly, when under the multi-client coalition, assumes that the coalition 21,2, ,4, n 1}. S E (sT,). The 2-client benefits between clients are
r(i, 1) = a:1 + E b
r(i,2) 02 + biz
(18)
r 1) = 0:n-1 + 8 bi-1
14

---

Algorithm 2: Dynamic Cooperative Evolution Algorithm
Input: K clients in set K., communication round benefit table with all states sh benefit vector u < 0, initial partition Jin < -[(),'' ( {K),. coalitions set $
Output: cooperative equilibrium
for p E K do
Calculate |1 0pp of p;
for - do Calculate r(p,9):
for S e S do
for k E S do
L Calculate benefit of client l in coalition S based on 11
r 0 then
Set Tin + {(1), (2). ... {K}}:
Perform Algorithm 1 to get n=:
else
Set Vin ..:; Update benefit table;
Perform Algorithm get
Set s < u(T))
Then for a client in S, the benefit can be expressed as the overall similarity between the aggregated model of the other models in excluding and the model of i.
Ue u mm)
9i) + e c0sgga 0z) aiaa |9 + ... + 0n 0gn Aa
V Volllen 112 + ... 0 119 1|
onbbil |10 + an bin 1n-
/n/101112 + 0 10 - 1/ H
(19)
E EpeSi(a) apaipllopll
E EpErT1)) 22
Epery(a Opbip 10
8
() 23 + H
where and I are defined in Eq. By mathematical induction, we can get for client in the coalition S == [1,2, ,,,,,,, ,d,
n.. With
rii,n) n) ain + E bin, we have
un (T cos (gagg Di) + cos(B 0²)
a1an1 + ann 19
Vafllgi|P ... a² |m
onbii |0: + a Din |0
+
... 02
/
12
(20)
Σ PPE apaipl Sppilllop11
E EpES)( a3l1g, 12
EpEr) apb1r110p11
+8*
1 Eppssmo 0
15

---

10
caunt tahle
cealifiee
coner
DSA
Round-1
D.AAS
3 @b)
8
5
110
E11
12
C1s
E11
e1s
Rag.
update parttion
count table
calike
count table
SiC -
Round-2
(12445)
= (009) (12A5)
61
Round-3
cxplii Each sound inavene and compan: s with arrerr Doffeent colors uproson no coepenitie nahiiondiis Chierss in coalition indional by the sim color and ontectal shert lines White reposen. singltion, mearing dade
61
10:00 00
Rar
equiitruun
wilh otlars.
Figure 5: Equilibrium forming process of 10 clients based on Merge-Blocking Algorithm.
B Detailed Description of the Algorithm
B.1 Dynamic Cooperative Evolution Algorithm
With the dynamic arrival of tasks, the equilibrium state also dynamic following a Markov process, which means the next equilibrium depends solely on the previous equilibrium. We use the dynamic cooperative evolution algorithm to evolve the new equilibrium each aggregation phase shown in Algorithm
B.2 Illustrate the Merge-Blockingg Algorithm with an Example
We offer 10 clients as example to further illustrate the process of achieving equilibrium in Fig.
according to the Algorithm [ on EMNIST-LTP settings. Initially, all client subsets are gener- ated as the coalition set S [(].....
(.f......99,, (9). 10. 1).... 10 9}}, and the initial partition is Tan = [{0], (1), (2), (3), (4), (5), (6), (7), (8), [9)], Tap = Tin- At the beginning of the first while loop (Round 1), when comparing with coalition {0.1} E S, the profitable transition met (i.e., all (u: (f0, 13) 2 us (Tap)) i i (0,
= and any(us (0, 1}) Z (i E {0, 1}) = 1), the original two coalitions {0} and {1} in the partition merged into the blocking coalition (BC) S = {0. 1}, and other coalitions remain unchanged, forming new Tap H0 1}. (2). (3). (4). (5). (6). (7), (8). {9}] at time eg. At e11, when compares with S S= 2). . the original 1,7) conforms the PT condition, so extracts 1 to cooperate with 2,8 forming (1, 2, (BC) and leaves {0,7] to form new uap = [11, 2, 2), {0, 7). (3, 6). (4, 5), (9)]- Then continue to traverse S E to compare. After each update, the count of coalitions is accumu- lated. If Tup update, the count of changed coalition becomes 0. After traversing S once, the coalition with the largest count is the stable coalition SC. as no BC for it appears. Therefore S is pruned by removing all coalitions containing clients which belong to SC. In next Rounds, Tup begins to traverse S E S again until there is no BC to update the partition, then up is combined with all previous SCs to obtain final equilibrium ~..
(PT) condition
B.3 Illustrate Dynamic Cooperative Evolution Results on EMNIST-LTP
As shown in Fig. 6 we list equilibrium states at the end round of each task phase on EMNIST-LTP dataset, and it can be seen that the coalition structure changes as the task changes, with clients of the same color forming a coalition. For example, at 4 there are coalitions and coalitions for 12. With the dynamic task flows, cooperative learning through dynamic coalition is necessary. Meanwhile, as the amount of tasks increases, clients tend form grand coalition acquire each other's information in order to recall the previous knowledge.
16

---

task
F2
K5
F2
C4
F2
22
client
Co
C3
C4
C5
C6
C7
Figure 6: Dynamic Cooperative Evolution EMNIST-LTP Dataset (c 0.8, À 0.2).
B.4 Time Complexity Analysis
Suppose that the number clients is K, number of cooperative states in the FCL system Bx and the number of coalitions is 2 1. The analysis of the time complexity is as follows:
(1) Formulating benefit table: In the initialization phase, we only need measure the overall similarity of 2-client structure, so the complexity of benefit calculation is O(3)). The complexity of calculating the size of K clients' models is (K)). The values obtained from the initialization can be directly calculated to form the benefit table. Go through all coalitions, each coalition has k clients, and total iteration is Σ k= 1 C K2K complexity is O(K**). Since the complexity of calculating the benefits of the multi-elient structure according to the derivation formula is O(1), so the total complexity is O(K²*). The greatest advantage of our benefit calculation method over other algorithms lies in the fact that we can calculate the individual benefits of rational clients under different groups, rather than only the collective benefits. Additionally, if we aggregate models to calculate the test accuracy on the local client as benefits, the total time complexity is O(K2* N) there are N test samples. In contrast, using coalitional affinity game and overall similarity greatly reduces the complexity of formulating benefit table.
Each iteration of merge-blocking algorithm needs to traverse all coalitions and compare the benefits of clients, therefore the complexity is also ((K2 The amount of computation is greatly reduced compared the complexity O 2 of traversing TPEF of all states in 3.3
(2) Achieving dynamic cooperative equilibrium:
We also list some algorithms using group aggregation for Federated Learning in Table and select representative metrics for contrast.
Table Compare the complexity of different group aggregation algorithms.
Algorithm
Benefit Calculation
Group Formation
Group's 's Number
Rational Optimal Salution
Dynamic Group
ClusterFL
O(K?) O(K
O(K*) O(K M2 TR3M) O(maxi K3, K2
2
FedGroup
M unlimited unlimited
Coalitional FL
O(K")
PFedSV
O(KKN)
DCFCL
O(K")
O(K2*)
V
V
ClusterFL quantifies benefits through pairwise similarity and employs Optimal Bipartition Algorithm to minimize inter-group similarity [7]: FedGroup decomposes all weights via Singular Value Decom position (SVD) into M vectors and applies K-means++ clustering over T iterations (33; Coalition FL utilizes EMDD-based linear combinations of data distributions with Accelerated Device Coalition Formation Algorithm (whose complexity matches ours when Imar = K, the maximum number of clients) 39 DFedSV forms coalitions for each client via top-k Shapley values at O(!KN)
FFrr fairness. here only list the benefits calculation under the 2-client structure, other algorithms do calculate benefits of multi-clients.
17

---

complexity for N test samples [8] Our algorithm shows following advantages: (1) provides game- theoretically optimal solution for rational clients, though with increased complexity compared to clustering ones 7 43 (2) dynamic, scale-unlimited coalition better adapts to continual learning, as evidenced by superior performance; (3) while maintaining computational efficiency through coalition affinity game and structured assumptions (additive/symmetri benefits), we quantify benefits for each client- a feature shared only with Coalition FL providing a reliable idea for incentives and benefit allocation. Our method's ' core objective of finding optimal client cooperation in federated continual learning inherently involves computational complexity, as the problem is NP-hard by nature. While method achieves optimal performance at relatively small scale, practical deployment for large scale necessitates approximating solution, which sacrifices theoretical optimality for computational feasibility, thereby becoming performance-cve tradeoff..
B.5 Boarder Impact
To achieve cooperation, all clients must share their model information. This process is facilitated by an impartial and authoritative third party, such as the industry association. The designated third party collects the client models after each round, then assesses the benefits in each state by comparing the overall similarity to determine equilibrium. The cooperative strategies are then published. Therefore, our framework promotes transparent and incentive-aligned cooperation among clients. At the same time, our framework can quantify benefit from each client in a coalition. In practice, such information can be utilized to either provide incentives or to impose charges on each client, to facilitate and enhance the foundation of the coalition.
C Implementation Details
C.1 Datasets
We construct a series of datasets comprising multiple federated clients, with each client possessing sequence of tasks. Suppose we use K denote the number of clients, T denote the number of tasks in each client, and C' to denote the number of classes in each task. We curate tasks by randomly selecting several classes from the datasets and sample part of the instances from these classes. Adhering to the principle of class incremental learning, there are no overlapped classes between any two tasks within a client.
EMNIST-LTP B]. The EMNIST dataset is a character classification dataset with 26 classes. contains 145600 instances of 26 English letters. The data contains upper and lower cases with the same label, making classification more challenging. To curate a dataset under LTP setting, we randomly sampled classes from the entire dataset for each client. The EMNIST-LTP dataset consists of 8 clients, with each client encompassing 6 tasks, each task comprising 2 classes (K = T = 6, C = 2).
EMNIST-shuflle (3]. In a conventional reshuffling setting, the task sets are consistent across all clients, while arranged in different orders. Therefore, with the same structure as EMNIST-LTP, we construct EMNIST-shuffle dataset with 8 clients, 6 tasks, and each task comprising 2 classes. While the 6 tasks of all clients are the same but in shuffled orders (K = 8, T = G,C 2).
CIFARI00 40). As a challenging image classification dataset, CIFARION 100 consists of low resolution images containing various objects and complex image backgrounds. We randomly sample 20 classes among 100 classes of CIFARIO0 100 as a task for each of the 10 clients, and there are 4 tasks for each client. For each class, we randomly sample 400 instances into the client dataset (K = 10, T = 4, C = 20).
MNIST-SVHN-F 9 41 (2). The dataset is constructed with MNIST [9. SVHN 41 and Fash- ionMNIST 42 Similar to MNIST, SVHN dataset serves as benchmark for digit classification
tasks, notable for its representation of real-world scenarios with complex backgrounds. We unify the labels of these two datasets. FaShionMNIST dataset is designed for clothing image classification. We set 10 clients in the mixed dataset, with each client containing 6 tasks, and each task has 3 classes. this mixed dataset, different tasks rely on different features. For example, shape features that are relevant to digit classification differ significantly from those that are important for classify- ing clothing items. Under centralized methods, it may result incredible knowledge interference (K 10, T = 6, C' 3).
18

---

C.2 Baselines
We compare our method with five baselines from FL, two baselines from CL. and six baselines from FCL. FL methods include basic centralized technique FedAvg. FedProx and SCAFFOLD for reducing heterogeneity interference, decentralized technique CFL for group aggregation, and personalized federated learning method Per-FedAvg. To control variables during local training, we incorporate knowledge distillation into all FL. baselines. CL methods are respectively combined with the FL methods (FedAvg. FedProx)) training global model while fighting catastrophic forgetting. The FCL methods focus on addressing the issues of catastrophic forgetting along with statistical heterogeneity.
Local. typical FL comparison method to achieve local training, without global aggregation. In order to control the experimental comparison, we add knowledge distillation to the local training.
FedAvg MT]. As representative FL method, FedAvg trains the models in each client with local dataset and averages their parameters to attain a global model.
FedProx 20 The algorithm is similar to FedAvg. While training local models, regularization term is employed to govern the proximity between the local parameters and the global parameters. This regularization term serves to effectively control the degree of deviation exhibited by the local models from the global model during the training process.
SCAFFOLD 21 addresses the issue of client drift by introducing control variates that help align local updates more closely with the global model. This reduces the divergence caused by non-III) data across clients, leading to faster and more stable convergence.
CFL 77. It is designed to optimize federated learning in enviromments with diverse client data distributions. CFL clusters clients into groups based on their data similarity and trains separate models for each group, allowing for personalized and accurate models while preserving privacy.
Per-FedAvg [0.. is an extension of FedAvg designed to enhance personalization in federated learning. Per-FedAvg focuses on producing a personalized model for each client by incorporating local fine-tuning. This approach balances the benefits of collaborative learning with each client's unique data characteristics.
PODNet 44 A CL method, it incorporates spatial-based distillation loss onto the feature maps of the classifier. This loss term serves to encourage the local models to align their respective feature maps with those of the previous model, thereby maintaining the performance previous tasks.
ACGAN-Replay [45] This CL algorithm employs a GAN-based generative replay method. The algorithm trains an ACGAN in the data space to memorize the distribution of previous tasks. While learning new tasks, the classifier is trained on new task data along with generated data from ACGAN.
As FCL algorithm, FLwF2T leverages the concept of knowledge distillation within the framework of federated learning. It employs both the old classifier from the previous task and the global classifier from the server to train the local classifier.
FLwF2T 26
The FCL algorithm extends the ACGAN- Replly method within the federated scenario, addressing the statistical heterogeneity issue with distillation loss.
FedCIL 28
GLFC [27]. In the FCL scenario, the algorithm exploits iiiillaationbbased method to alleviate the issue of catastrophic forgetting from both local and global perspectives.
AF-FCL 29 proposes adaptive forgetting mechanism that dynamically adjusts knowledge retention policies to address catastrophic forgetting in heterogeneous federated learning scenarios.
AFCL 6 introduces an asynchronous training paradigm with adaptive synchronization to enable efficient continual learning across heterogeneous federated devices while mitigating forgetting.
FPPL 46 introduces a novel federated prototype learning framework that simultancously addresses catastrophic forgetting and data heterogeneity through efficient prototype propagation and local consistency regularization.
C.3 Metrics
We use the metries of accuracy and average forgetting for evaluation works Suppose at is the test set accuracy of the t-th task after learning the i-th task in client k.
19

---

Table 5: Average accuracy on CIFARI00 when K = 8, = 6, C = 10.
Model
CIFARI00
FedAvg
19.5
FedProx
20.1
SCAFFOLD
20.3
CFL
20.5
Per-FedAvg
29.6=1.1
PoDNet+FedAvg
21.3
PODNe++FedProx
21.6=0.4
ACGAN+FedAvv
19.5
ACGAN+FedProx
19.6:0.2
FLwF2T
21.5
FedCIL
19.640.3
GLFC 19.9
DCFCL
31.4
Average Accuracy. We evaluate the performance of the model on all tasks in all clients after it finish learning all tasks. By using a weighted average, we calculated the accuracy for all seen tasks across all clients, with the number of samples in each task serving as the weights:
Average Accuracy
Σk
E
T ag
ni-
(21)
1= R
k=
1=1
This approach allows to account for variations in task difficulty and ensure a fair evaluation across different tasks and clients.
Average Forgetting. The metric of average forgetting assesses the extend of backward transfer during continual learning, quantified as the disparity between the peak accuracy and the ending accuracy of each task. We also use a weighted average when calculating average forgetting:
K
Average Forgetting
max (E(1....TT-
(ak
0 T.A)
ni.
(22)
ΣR Σ
E(=1
T
SE
C.4
Optimization
The Adam optimizer is employed for training all models. For all experiments except for CIFAR 100, a learning rate of le-4 is utilized, with a global communication round of 60, and local iteration of 100. We set learning rate 1e-3, the global communication round as 40, and local iteration 400 for CIFARION 100. Other parameters include weightdecay = le 5, betal = 0.9, beta2 0.999, For training. mini-batch size of 64 adopted. The number of generated samples in an iteration aligns with this mini-batch size. We report the mean and standard deviation of each experiment, conducted five times with different random seeds.
C.5 Model Architectures
In the case of CIFAR100, we utilize the feature extractor of a ResNet-18 47 as ha and hy comprises two FC layers, both with 512 units. For other datasets we adopt three-layer CNN followed by an FC layer with 512 units as ha. The channel numbers of the convolutional layers are [64, 128, 256]. And hg represented by an FC layer. The outputs ht belong to 512, All the FC layers employed in the architectures consist of 512 units. The convolutional layers and FC layers are followed by Leaky ReLU layer. Another FC layer serves he and operates as the classification head.
C.6 Devices
In the experiments, we conduct all methods on local Linux server that has two physical CPU chips (Intel(R) Xeon(R) CPU E5-26440 v4 @ 2.40GHz) and 32 logical kernels. All methods are implemented using Pytorch framework and all models are trained on GeForce RTX 2080 Ti GPUs.
20

---

D Additional Experimental Results
D.1 More Complex Scenario
We conduct experiments CIFARI00 in more challenging setting. We randomly sample 10 classes among 100 classes CIFARI00 as a task for each of the 8 clients, and there are tasks for each client (K 8, 6, C 10). For each class, randomly sample 400 instances into the client dataset. Therefore, each client possesses more tasks with fewer samples per task.
As shown in Table 5 method achieves the highest average accuracy among the evaluated approaches. While the CL approach emphasizes retaining knowledge from previous tasks and the traditional FCL approach focuses centralized aggregation to ensure that client knowledge is utilized totally, these methods can sometimes have a negative influence by indiscriminately aggregating information. In contrast, client coalitions through dynamic cooperative learning. This approach aggregates clients with similar tasks, mitigating forgetting within local coalitions, especially when data heterogeneity among clients is significantly strong. Therefore, compared to established baselines, our method achieved the highest average task test accuracy.
proposed method utilizes decentralized federated aggregation to form
D.2 Communication Cost
To reduce communication overhead, cache the model information from the previous aggregation round on both the client and the third party. This allows gradient information to be calculated by model differences, only model parameters need to be transmitted in each communication round.
We list the communication cost Table. of different methods across four datasets. C2S represents client-to-serve cost, S2C is cerver-to-client cost. The results demonstrate that DCFCL achieves optimal communication efficiency in all datasets, matching the performance of the most basic FedAvg and FedProx methods while significantly outperforming improved approaches that require additional communication overhead (such as SCAFFOLD and CFL, which typically double the communication volume in the C2S direction). It is particularly noteworthy that although methods like ACGAN and FedCIL enhance model performance by incorporating generative models, they all introduce varying degrees of increased communication costs. In contrast, DCFCL ensures performance improvements while completely avoiding additional communication burdens.
Table 6: The client to server(C2S) and sever to client(S2C) communication cost(GB) during the whole training process.
CIFAR100
EMNIST-LTP
EMNIST-shuffle
MNIST-SVHN-F
Model
C2S
S2C
C2S
S2C
C2S
S2C
C2S
S2C
FedAvg
10.400
10.400
4.056
4.056
4.056
4.056
7.260
7.260
FedProx
10.400
10.400
4.056
4.056
4.056
4.056
7.260
7.260
SCAFFOLD
20.800
10.400
8.112
4.056
8.112
4.056
14.520
7.260
CFL
20.800
10.400
8.112
4.056
8.112
4.056
14.520
7.260
Per-FedAvg
10.400
10.400
4.056
4.056
4.056
4.056
7.260
7.260
PODNet+FedAvg
20.800
10.400
8.112
4.056
8.112
4.056
14.520
7.260
PODNet+FedProx
20.800
10.400
8.112
4.056
8.112
4.056
14.520
7.260
ACGAN+FedAvg
10.523
10.400
4.093
4.056
4.093
4.056
7.440
7.260
ACGAN+FedProx
10.523
10.40
4.093
4.056
4.093
4.056
7,440
7.260
FedCIL.
20.800
10.400
8.112
4.056
8.112
4.056
14.520
7.260
AF-FCL
20.800
10.400
8.112
4.056
8.112
4.056
14.520
7.260
AFCL
10.420
10.400
4.062
4.056
4.062
4.056
7.260
7.260
FPPL
10.420
10.400
4.062
4.056
4.062
4.056
7.260
7.260
DCFCL
10.400
10.400
4.056
4.056
4.056
4.056
7.260
7.260
D.3 Mitigation of Catastrophic Forgetting
We compare the forgetting rate in Table. 7 to further demonstrate the effectiveness. The results clearly demonstrate DCFCL's superior performance in mitigating catastrophic forgetting, achieving
21

---

Table 7: The average forgetting rate (%) on 4 datasets.
Model
CIFARI00
EMNIST-LTP
EMNIST-shuffl
MNIST-SVHN-FF
FedAvg
8.640.9
24.0+0.6
9.6.0.9
25.640.6
FedProx
8.440.6
23.840.7
8. 8.1
24.9
SCAFFOLD
8.2.0.7
19.240.3
8.2+0.7
22.1
CFL
8.9
19.8+0.6
9.4+0.6
24.4
Per-FedAvg
8.7
19,440.5
7.840.6
21.9+0.7
PODNct+FedAvg
8.6+0.6
15.5+0.7
7.3+0.9
21.3+0.3
PODNet+FedProx
7.5
14.3+1.2
6.0
20.640.8
ACGAN+FedAAv
6.440.7
14.34u.5
6.5
20.0
ACGAN+FedProx
6.2.104
12.4
6.1 6.1
19.7
FedCIL
6.5
10.44u.4
6.4
19.7
AF-FCL
4.940.9
7.9 7..4
4.2
7.5
AFCL
6.340.5
10,540.9
5.7
11.3
FPPL 6.9
11.640.3
7.4
13.2
DCFCL
4.7.1.9
8.910.6
4.2
6.9 6.9_0.7
the lowest forgetting rates on 3 datasets. This represents reduction compared to baseline methods like FedAvg and FedProx. DCFCL's dynamic collaboration mechanism achieves significantly better retention without requiring additional memory buffers complex architectural modifications. These consistent improvements across diverse datasets underscore DCFCL's robustness in preserving learned knowledge while accommodating new information.
Before
Centralized
w/o MBA DCFCI
Average Forgetting Rate (%%
Equilibrium, Time (min)
40
20
7
8
9
10
20
30
40
50
Client
Number Clients
(4)
(b)
Figure 7: Comparison of average forgetting of each client on MNIST-SVHN-F(leff) The equilibrium achieving time when the number of clients increases(right).
We also list forgetting mitigation of each client to illustrate the benefits of cooperation comparing no aggregation (local), centralized aggregation and decentralized (DCFCL) methods, as shown in Fig. Aa). The local method (yellow) show high forgetting rate of 60%-95%. After adopting centralized aggregation (blue), the forgetting rate significantly decrease to 10%-36% indicating that the aggregation between clients can promote the knowledge recall of different clients respectively, there is still room for optimization. The decentralized dynamic cooperation method (orange) demonstrate better results, stably controlling the forgetting rate below 10% (3.49%- 9.70%). It is particularly worth noting that DCFCL maintains the lowest and most stable forgetting rate on all clients, significantly reducing the differences in forget rates among clients.
D.4 Computation Cost
We present Fig. (Tbb) show the computation time of equilibrium as the client number increases. comparing it to the method without perge-blookingg algorithm (MBA). As the number of clients grows from 10 to 50, the conventional approach without MBA shows exponential time escalation, highlighting scalability issues. In contrast, DCFCL maintains polynomial time complexity, with computation times increasing only from 0.1 min 18. 733 min, with the gap widening as system scale grows. This demonstrates DCFCL's advantage in large-scale deployments.
22

---

Table 8: The run-time consumption comparisons ((min) on 4 datasets.
Model
CIFARI00
EMNIST-LTP
EMNISST-huufle
MNIST-SVHN-F
FedAvg
238
22
21
29
FedProx
246
26
24
SCAFFOLD
265
38
37
45
CFL
294
34
35
Per-FedAvg
287
32
28
32
PODNct+FodAvg
252
35
34
49
PODNet=FedProo
253
37
39
51
ACGAN+FedAvg
312
85
149
ACGAN+FedProx
315
89
82
152
FedCIL
322
93
90
172
AF-FCL
302
62
60
126
AFCL
277
34
32
44
FPPL
253
32
31
45
DCFCL
294
39
39
48
Table. 8 compares runtime performance across four datasets. DCFCL shows competitive runtime (48-294 minutes), similar CFL and SCAFFOLD. Generative methods (ACGAN and FedCIL) incur higher overhead (up to 322 minutes on CIFAR 100), while traditional methods like FedAvg and FedProx are faster (32-238 minutes) but may sacrifice performance. DCFCL strikes balance between efficiency and learning ability, with moderate runtime costs across datasets.
23

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Based on the abstract and introduction, the three main claims made in the paper reflect the contributions and scope of the research, which focuses introducing decentralized dynamic cooperative framework for federated continual learning. All of these points addressed in the main text.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The main limitations of the work, well as future directions that might address some of these limitations, are laid out in the conclusion portion of the paper.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
24

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: The assumptions main theorem is laid out in the main text, while the derivation process and proof details placed in the supplementary material space.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Yes, the paper provides detailed experimental settings, including the datasets used, client and task configurations, the specific methods for comparison, and the eval- uation metrics. It also includes ablation studies and results with different parameter settings to support the validity of its claims. These details are sufficient for repro- ducing the main experimental results and verifying the conclusions. Meanwhile, we also provide the complete code required for the reproduction, which is available at: wwwp://wwoooooo.oooooeneeeeeeeie
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
25

---

(d)
The authors should provide scripts to reproduce all experimental results for the new reproducible, they We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Yes, the paper provides a link to the code repository for open access, which contains scripts reproduce all experimental results for the new proposed method and base- lines. The supplemental material likely contains more information on the exact procedure for reproducing the results, such dataset settings and baselines.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
Sublic/guides. Dtggliicguudee//odeeuubbmss wwweeoobiiiiiiiioooiiiiiiiooeeie
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/pips..cc/pulic//guu/ http:/uddiessseeeeee /
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Yes, we provide sufficient details in the supplemental material regarding the training and testing setup, including data splits and the use of the optimizer with specified hyperparameters.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
26

---

Yes, we measure the mean and standard deviation of each experiment to report error bars for the experimental results, conducted three times with different random seed. Therefore, the robustness and consistency of our method are reliably verified. Guidelines:
Justification:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative етror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, the supplemental material provides details the computational resources used, including the computation time and communication cost. Meanwhile, the device information for running the experiment is provided at the end of the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Yes, we believe that our work conforms to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
27

---

10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed in boarder impact in Appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymeent of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out, For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: No such models datasets are involved.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We use the code of baselines, framework of PyTorch and open-source datasets, and cite them in the main text.
28

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No such assets are introduced.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: No such experiments or datasets are involved.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
29

---

Justification:
We have no human participants in our study.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality the research, declaration is not required.
Answer: [NA]
Justification: We don't involve LLMs any important, original, or non-standard compo- nents.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
30


Reviews:
[{"review_id": "ScyXWkpauH", "reviewer": "Reviewer_u9Pu", "summary": "This paper proposes a federated continual learning method that estimates client benefits in cooperative coalitions by estimating task similarity and using a coalitional affinity game. In addition, this paper designs an algorithm to efficiently find and update the cooperative equilibrium of clients.", "questions": "1. Why is KL loss needed to achieve a better estimation of the coordination ability between tasks? In order to learn the task, the model deviates more from the previous round model, which may affect the aggregation effect, but this article does not explain how it is related to identifying cooperators.\n\n\n2. In the fields of FL, CL, and FCL, there are many methods for estimating task similarity and selecting clusters. How to explain that the method proposed in this article is better than them?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The motivation of this paper is clear, and an aggregation algorithm with certain theoretical guarantees is proposed to select the scope of aggregation for each task, effectively avoiding the knowledge conflict between tasks.\n\n2. This paper conducts relatively sufficient experiments on multiple data sets, which can illustrate the effectiveness of the proposed algorithm in resolving conflicts between tasks.\n\nWeaknesses:\n1. In the related work section, the reviewer believes that there are currently a large number of methods based on pre-trained models and parameter-efficient fine-tuning in CL and FCL. Although this article focuses on the effect of learning from scratch, it is necessary to discuss such methods in related work and explain the advantages of learning from scratch.\n\n2. In the field of continual learning, forgetting is an indicator often used to measure the ability of an algorithm to avoid catastrophic forgetting. Because the learning capabilities of different algorithms may vary, the reviewer believes that it is necessary to provide a forgetting indicator.", "was_revised": false, "final_justification": "", "rebuttal": "**Response to W1**:\n\n****Answer:**** We thank the reviewer for their insightful suggestion. While our current work investigates continual learning from scratch to analyze task coordination in a controlled setting, we acknowledge the growing importance of pre-trained models (e.g., LLMs, vision backbones) [6] and parameter-efficient tuning (e.g., adapters, LoRA) [7] in practical FCL. Here we concisely address this concern from three perspectives:\n1. **Method Comparision**: While these methods benefit from strong initialization and existing knowledge, leading to reduced training costs and improved early-stage performance, our approach concentrates on learning from scratch. This allows us to directly study coalition-based knowledge retention and generalization dynamics in federated continual learning without relying on pre-trained priors. Notably, our method is complementary to PEFT-based approaches and could potentially be integrated with them in future work.\n2. **Revised Version**: In the revised version, we will add a dedicated paragraph in the Related Work section discussing these approaches. \n3. **Method Extension**: To demonstrate our framework's adaptability to pre-trained foundation models, we conducted preliminary experiments applying our DCFCL approach with PEFT. Specifically:Replaced local models with a pre-trained ViT-B/16 backbone (iBOT-21K). Implemented adapter-based PEFT, where only PEFT parameters are updated and transmitted. Maintained our original coalition similarity computation using PEFT parameter embeddings. Preserved our benefit calculation and coalition evolution framework. Compared against FedAvg+adapter [6] and ClusterFL+adapter [8] baselines on CIFAR100. Our experimental results are as follows\n\n|Method|Accuracy (↑)|Forget (↓)|\n|-|-|-|\n|FedAvg+adapter|69.8|6.9|\n|Cluster+adapter|73.1|7.2|\n|DCFCL+adapter|77.5|4.2|\n\nAs shown in the Table, **DCFCL+Adapter** achieves the best accuracy and the lowest forgetting rate on CIFAR-100. These results validate the generalizability of our mechanism in PEFT scenarios.\n\n---\n\n**Response to W2**: \n\n****Answer****: Thank you for pointing this out. In our experiments, we follow recent works [9,10] and use average accuracy and average forgetting as evaluation metrics. Let $\\alpha_k^{t,i}$ denote the test accuracy on task $i$ after learning task $t$ in client $k$. Average forgetting measures backward transfer, defined as the difference between peak and final accuracy of each task. A weighted average is applied in its calculation:\n\\begin{equation}\n\\text{Average Forgetting} = \\frac{1}{\\sum_{k=1}^{K}\\sum_{i=1}^{T-1} n_k^i} \\sum_{k=1}^{K}\\sum_{i=1}^{T-1} \\max_{t \\in \\{1,\\cdots,T-1\\}} (a_k^{t,i} - a_k^{T,i}){n_k^i}\n\\end{equation}\nIn our revision, we will supplement this point in the experimental section.\n\n---\n**Response to Q1**\n\n****Answer:****  Thank you for your valuable comment. We would like to clarify a possible misunderstanding regarding the use of **KL loss** in our method. \n1. **Clarification**: Our method does not rely on KL divergence to measure task coordination. Instead, knowledge distillation is used as an auxiliary tool to preserve feature consistency during continual learning (Section 4.1). As new tasks alter the classifier, it becomes harder to identify cooperators for knowledge recall. By maintaining feature space consistency, we enable efficient cooperator matching using existing model information, with minimal communication overhead.\n2. **Empirical Validation**: We experimentally verify that our method, with KD as an auxiliary tool, better mitigates catastrophic forgetting than naive KD. Even applying KD to all clients fails to achieve optimal performance without our framework, as shown in Section C.2, line 530 (*\"To control variables during local training, we adopted the same KD method into all FL baselines, which still obtained suboptimal results.\"*).\n2. **Relation to Identifying Cooperators**: We use knowledge distillation between the classifier’s current and past outputs to maintain feature space consistency and prevent drift during continual learning. This helps retain discriminative features from previous tasks, enabling accurate identification of similar clients without additional model exchange. As shown in **Fig.2**, clients choose different cooperators at each task stage based on the **trade-off** between acquiring new knowledge and retaining prior learning. KD helps clients find suitable partners to preserve prior knowledge. In short, distillation is not for coordination estimation but for **preserving useful representations** to support **similarity-based** cooperation under dynamic, heterogeneous conditions.\n\n---\n**Respond to Q2**\n\n****Answer:****  Thank you for raising this. Based on our investigation, many existing works estimate task similarity via the following several mainstream metrics:\n\n1. **L2-based**: the $l_2$ distance across client models. [1]\n2. **Gradient-based**: only use the cosine similarity of client gradients.[2,3]\n3. **EDC**: truncate gradients onto key directions via Singular Value Decomposition and computes similarity in this reduced space. [4]\n4. **FC-based**: the cosine similarity between client classifier parameters.[5]\n\nUnder our experimental settings, we only replace the **overall similarity** part. The experimental results are as follows:\n\n|Method| EMNIST-LTP| EMNIST-shuffle|CIFAR100| MNIST-SVHN-F|\n|-|-|-|-|-|\n|  |Accuracy|Forget|Accuracy|Forget|Accuracy|Forget|Accuracy|Forget|\n|Grad-based|45.3|14.8|73.7|15.7|32.6|10.3|55.5|17.1|\n|EDC|49.8|10.4|72.9|13.6|32.9|8.2|60.4|12.2|\n|L2-based|50.5|12.4|73.2|12.0|33.7|8.4|65.8|14.2|\n|FC-based|51.4|**8.8**|77.5|7.6|37.5|6.8|62.5|10.5|\n|DCFCL|**52.5**|8.9|**78.3**|**4.2**|**40.4**|**4.7**|**66.7**|**6.9**|\n\nUsing gradient-based methods alone leads to severe forgetting, as gradient similarity struggles to identify models aligned with prior tasks, hindering knowledge retention. EDC reduces computation by truncating the gradient matrix and approximating similarity but yields suboptimal performance. In contrast, our overall similarity accurately identifies cooperators and evaluates benefits, achieving the best results in both accuracy and forget rate.\n\n[1] M. Xie, G. Long, T. Shen, T. Zhou, X. Wang, and J. Jiang, “Multi-center federated learning,” arXiv preprint arXiv:2005.01026, 2020.\n\n[2]  Sattler, F., Müller, K.-R., and Samek, W. Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints. IEEE Transactions on Neural Networks and Learning Systems, 32(8): 3710-3722, 2021. doi: 10.1109/TNNLS.2020.3015958.\n\n[3] Li, Z., Lin, T., Shang, X., & Wu, C. (2023). Revisiting Weighted Aggregation in Federated Learning with Neural Networks. arXiv preprint arXiv:2302.10911.\n\n[4] M. Duan, D. Liu, X. Ji, R. Liu, L. Liang, X. Chen, and Y. Tan, \"FedGroup: Efficient Federated Learning via Decomposed Similarity-Based Clustering,\" in Proc. IEEE Int. Symp. Parallel Distrib. Process. Appl , 2021, pp. 228-237.\n\n[5] Wu, S., Jia, Y., Liu, B., Xiang, H., Xu, X., & Dou, W. (2025). PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers. Proceedings of the AAAI Conference on Artificial Intelligence, 39(20), 21572-21580.\n\n[6] Cai, D., Wu, Y., Wang, S., Lin, F. X., and Xu, M. FedAdapter: Efficient Federated Learning for Modern NLP. arXiv preprint arXiv:2205.10162, 2023.\n\n[7] Yi, L., Yu, H., Wang, G., Liu, X., and Li, X. pFedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning. arXiv preprint arXiv:2310.13283, 2024.\n\n[8] Sattler, F., Müller, K.-R., and Samek, W. Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints. IEEE Transactions on Neural Networks and Learning Systems, 32(8): 3710-3722, 2021. doi: 10.1109/TNNLS.2020.3015958.\n\n[9] Wuerkaixi, A., Cui, S., Zhang, J., Yan, K., Han, B., Niu, G., Fang, L., Zhang, C., and Sugiyama, M. Accurate Forgetting for Heterogeneous Federated Continual Learning. In The Twelfth International Conference on Learning Representations (ICLR), 2024.\n\n[10] Qi, D., Zhao, H., and Li, S. Better Generative Replay for Continual Federated Learning. In The Eleventh International Conference on Learning Representations (ICLR), 2023."}, {"review_id": "1qJZY05XlE", "reviewer": "Reviewer_88x1", "summary": "This paper presents a novel decentralized dynamic cooperation framework (DCFCL) to address the challenges of temporal and spatial catastrophic forgetting in Federated Continual Learning (FCL). The authors formulate client cooperation using coalitional game theory, where each client dynamically joins coalitions based on estimated mutual benefit, evaluated via an overall similarity metric combining gradient coherence and model similarity. A merge-blocking algorithm and dynamic cooperative evolution strategy are proposed to ensure convergence to a cooperative equilibrium in each aggregation round. Extensive experiments across several datasets demonstrate the superiority of the proposed method over existing FL, CL, and FCL baselines.", "questions": "as above", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n1. The paper introduces a novel application of cooperative game theory in the FCL context, enabling clients to dynamically form non-overlapping coalitions. This design effectively balances knowledge retention and acquisition under data heterogeneity.\n2. The proposed overall similarity metric, which combines gradient and model-level coherence, is both theoretically grounded and computationally efficient. The merge-blocking and dynamic evolution algorithms are well-designed to support convergence to cooperative equilibrium.\n3. The method is rigorously evaluated on four benchmark datasets, including highly heterogeneous and multimodal settings. The authors also conduct thorough ablation studies and parameter sensitivity analyses to demonstrate the individual contributions of key components.\n\nWeakness\n1. In the Introduction, the authors claim that spatial catastrophic forgetting is caused by aggregation. However, the subsequent preliminary experiments demonstrate that federated aggregation helps alleviate forgetting, which appears contradictory. Could the authors clarify this seeming inconsistency?\n\n2. The manuscript does not provide a clear definition of Personalized Federated Continual Learning (FCL). What exactly distinguishes personalized FCL from decentralized FCL? A clearer conceptual distinction between the two is necessary.\n\n3. Regarding the client grouping strategy for utility computation: is it based on exhaustive enumeration or another approach? Given that the benefit comparison involves dynamic equilibrium, which can be computationally expensive, are there any strategies proposed to reduce time complexity?\n\n4. Does the proposed method involve storing state spaces? If so, what is the associated space complexity?\n\n5. In Table 3, two important hyperparameters significantly affect model performance. The selection of these hyperparameters is critical for practical deployment. Additionally, on the EMNIST-shuffle dataset, model performance appears more sensitive to these hyperparameters. Is there any analysis or discussion provided on this issue?", "was_revised": true, "final_justification": "The authors have further elaborated the concepts, time complexity, space complexity, and hyperparameter robustness in the paper, and have basically answered the review comments. Considering the overall contribution of the paper, I maintain the  previous score.", "rebuttal": "## Reviewer 88x1\n\nWe would like to thank the reviewer for the useful comments. Below are our responses to the comments in **Weaknesses**.\n\n---\n\n#### ***Weakness 1: In the Introduction, the authors claim that spatial catastrophic forgetting is caused by aggregation. However, the subsequent preliminary experiments demonstrate that federated aggregation helps alleviate forgetting, which appears contradictory. Could the authors clarify this seeming inconsistency?***\n\n****Answer:**** Thanks for the useful comments. We will clarify our statement from following aspects.\n\n1. **Federated Aggregation Benefit**: We acknowledge that federated aggregation can mitigate forgetting, as knowledge from other clients helps recall prior tasks. This is illustrated in Section 2, Paragraph 2, via Fig. 1(a)(b), highlighting FCL’s value and motivating the use of central aggregation [1,2].\n\n2. **Critical Limitation**: However, we question whether aggregating all models is always beneficial. We argue that blind aggregation is suboptimal—especially under strong spatial heterogeneity, which may cause severe knowledge interference, termed spatial catastrophic forgetting in prior work [3,4].\n\n3. **Empirical Validation and Motivation**: As shown in Paragraph 3 and Fig. 1\\(c\\), we empirically verify this concern. We emphasize that selective and cooperative aggregation is essential for personalized continual learning, motivating the design of our DCFCL framework.\n\n[1] Yoon, J., Jeong, W., Lee, G., Yang, E., & Hwang, S. J. (2021). Federated continual learning with weighted inter-client transfer. In Proceedings of the 38th International Conference on Machine Learning (pp. 12073-12086). PMLR.\n\n[2] Shenaj, D., Toldo, M., Rigon, A., & Zanuttigh, P. (2023). Asynchronous federated continual learning. In *2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. IEEE.\n\n[3] X. Yang, H. Yu, X. Gao, H. Wang, J. Zhang and T. Li, \"Federated Continual Learning via Knowledge Fusion: A Survey,\" in IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 8, pp. 3832-3850, Aug. 2024.\n\n---\n\n#### ***Weakness 2: The manuscript does not provide a clear definition of Personalized Federated Continual Learning (FCL). What exactly distinguishes personalized FCL from decentralized FCL? A clearer conceptual distinction between the two is necessary.***\n\n****Answer:**** Thanks for the useful comments. Here we give the definition of two concepts.\n\n1.**Personalized FCL**: Personalized FCL is a scenario where clients aim to preserve their individual task histories and learn models tailored to their own data distributions across time. In PFCL, clients can share models to facilitate their own performance [1,2].\n\n2.**Decentralized Learning**: Decentralized learning refers to the absence of a central server in communication topology [2,3]. Clients spontaneously aggregate models and don't rely on central server. Our work emphasizes the personalized continual learning objective under decentralized settings.\n\n3.**Framework Objectives and Rationale**: Since our personalized FCL framework is based on dynamic cooperation, the objective is to optimize the performance of each rational client rather than a global objective as shown in Eq.1.:\n\\begin{equation}\n\\underset{\\theta_k^t}{\\operatorname*{\\operatorname*{argmin}}}\\underset{\\mathcal{D}_k^t}{\\operatorname*{\\operatorname*{E}}}[L_k^t(\\mathcal{D}_k^t,\\theta_k^t)]\n\\end{equation}\nwhich naturally requires no central architecture. Thus, we argue decentralized topology better suits personalized FCL than centralized, as discussed in Section 1 (Paragraphs 2-3).\n\n4. **Revision Plans**: In the revised version, we will make two definitions more precise in Section 1.\n\n[1] J. Xie, C. Zhu, and S. Li, \"FedMeS: Personalized Federated Continual Learning Leveraging Local Memory,\" arXiv preprint arXiv:2404.12710, 2024.\n[2] S. Li, T. Zhou, X. Tian and D. Tao, \"Learning to Collaborate in Decentralized Learning of Personalized Models,\" 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, 2022.\n\n---\n\n#### ***Weakness 3: Regarding the client grouping strategy for utility computation: is it based on exhaustive enumeration or another approach? Given that the benefit comparison involves dynamic equilibrium, which can be computationally expensive, are there any strategies proposed to reduce time complexity?***\n\n****Answer:**** Thanks for the comments. Our coalition formation avoids exhaustive enumeration. Instead of exploring all grouping states, we iteratively merge coalitions under the **PT condition** (Profitable Transition: merging occurs only when clients gain higher benefits). For complexity reduction, we employ these strategies:\n\n1. **Benefit Calculation via Coalitional Affinity Game**: To avoid prohibitive computation/communication costs from exhaustive local aggregation, we propose overall similarity to quantify pairwise client benefits. Using coalitional affinity game, we prove multi-client benefits can be derived from 2-client relationships via function $f(\\cdot)$ (Appendix A). This reduces complexity from $O(\\sum_{k=1}^K C_K^k)$ to $O(C_K^2)$ (Appendix B.4).\n\n2. **Merge-Blocking Algorithm**: Our iterative algorithm (Section 4.2) avoids traversing the exponential partition space. While each iteration remains $O(K2^K)$, it significantly improves over the $O(B_K 2^K)$ complexity of full state-space traversal.\n\n3. **Pruning in Merge-Blocking**: Redundant subset comparisons are eliminated each round. Empirical results show fast convergence regardless of client count, with formation times substantially reduced versus non-optimized approaches.\n\n\n| Number of clients | 10| 20| 30| 40| 50 |\n|:-------------------:|:-------:|:-------:|:--------:|:-------:|:-------:|\n| Time (min)          | 0.116  | 2.715  | 8.279  |12.526  | 27.15  |\n| Iterations        | 2      | 2      | 3       | 3  | 5  |\n| Time (min)-w/o MBA         | 1.438  | 8.504  | 24.733  | 39.293  | 54.596  |\n\n---\n\n#### ***Weakness 4: Does the proposed method involve storing state spaces? If so, what is the associated space complexity?***\n\n****Answer:**** We thank the reviewer for raising this concern. Our method avoids explicit storage of the entire state space by maintaining only a small cache of previous-round model states and utility values for current candidate coalitions. The space complexity analysis shows this design achieves linear scaling: each client requires $O(d)$ local storage for knowledge distillation (significantly less than rehearsal-based CL approaches that store prior task data/generators), while the third-party coordinator maintains $O(2Kd)$ space for current/previous models of all clients (detailed in Section 4.3 and Appendix B.1). We will further clarify this space complexity analysis in the final manuscript.\n\n\n---\n\n#### ***Weakness 5: In Table 3, two important hyperparameters significantly affect model performance. The selection of these hyperparameters is critical for practical deployment. Additionally, on the EMNIST-shuffle dataset, model performance appears more sensitive to these hyperparameters. Is there any analysis or discussion provided on this issue?***\n\n****Answer:**** We thank the reviewer for raising this concern. To further illustrate, we conduct an extended sensitivity analysis in the following tables. \n\n**Sensitivity Analysis of λ and ε on MNIST-SVHN-F**\n\n| λ\\ε | 0.0   | 0.2   | 0.4   | 0.6   | 0.8   | 1.0   |\n|:---:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|\n| **0.0** | 60.5 | 62.1 | 63.8 | 63.1 | 61.7 | 60.9 |\n| **0.2** | 61.2 | 61.4 | 64.9 | 62.4 | 62.5 | 61.3 |\n| **0.4** | 62.0 | 64.2 | 65.7 | 64.7 | 63.2 | 62.1 |\n| **0.6** | 60.1 | 64.8 | 66.1 | 64.0 | 63.9 | 62.8 |\n| **0.8** | 63.8 | 64.3 | 66.5 | 65.2 | 64.3 | 61.2 |\n| **1.0** | 64.5 | 65.9 | **66.7** | 65.8 | 64.9 | 62.5 |\n\nWe compared the maximum fluctuation in accuracy when one value was fixed and the other changed.\n$\\lambda=0.4,max(Range_\\varepsilon) =3.6$\n$\\varepsilon =0.2,max(Range_\\lambda) =3.8$\n\n**Sensitivity Analysis of λ and ε on EMNIST-Shuffle**\n| λ\\ε | 0.0   | 0.2   | 0.4   | 0.6   | 0.8   | 1.0   |\n|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|:---------:|\n| **0.0** | 70.2 | 73.2 | 81.9 | 73.6 | 74.1 | 64.2 |\n| **0.2** | 75.9 | 78.3 | 80.2 | 70.4 | 71.4 | 72.2 |\n| **0.4** | 78.7 | 78.9 | 82.9 | 71.2 | 70.9 | 70.6 |\n| **0.6** | 78.0 | 79.6 | 82.7 | 73.7 | 71.1 | 71.0 |\n| **0.8** | 75.2 | 79.7 | 81.7 | 74.2 | 73.2 | 74.2 |\n| **1.0** | 73.1 | 81.3 | **84.5** | 80.7 | 76.9 | 74.5 |\n\nSimilarly, for the emnist-shuffle dataset:\n$\\lambda =0.0,max(Range_\\varepsilon) =17.7$\n$\\varepsilon =1.0,max(Range_\\lambda) =10.3$\n\nEMNIST-shuffle shows greater sensitivity to $\\lambda$ and $\\varepsilon$ due to extreme class imbalance, requiring larger $\\varepsilon$ to better leverage global model information for collaborator identification - highlighting the critical importance of proper similarity weight settings for highly heterogeneous datasets."}, {"review_id": "sRwj4NJbUc", "reviewer": "Reviewer_ckij", "summary": "This paper presents DCFCL, a decentralized framework for Federated Continual Learning where clients form dynamic coalitions via game theory to mitigate catastrophic forgetting. To efficiently estimate cooperation benefits without expensive trials, it uses a proxy metric called \"overall similarity,\" which combines gradient and model similarity as $\\cos(g_i, g_j) + \\epsilon \\cdot \\cos(\\theta_i, \\theta_j)$. A merge-blocking algorithm is then used to find a stable coalition equilibrium.", "questions": "1.The $O(K2^K)$ complexity makes your method impractical for large-scale FL. How do you propose to solve this for K > 100? Are there viable approximation schemes? \n2.Your \"overall similarity\" proxy is a heuristic. Can you provide a concrete example (synthetic or otherwise) where it fails and leads to a sub-optimal coalition compared to the ground-truth benefit?\n3.Your framework describes a complex dynamic system. Can you provide a proof or at least a strong argument for the convergence of the overall client model training process across tasks? Without it, the stability of the entire learning process is in question.\n4.Justify the decision to re-evaluate coalitions every round. How does it compare in terms of final accuracy versus total computation time against a less frequent update schedule (e.g., once per task)?", "limitations": "1.The exponential computational complexity ($O(K2^K)$) is a hard barrier to scalability in systems with many clients. \n2.The framework introduces several critical hyperparameters (e.g., $\\epsilon$ for similarity weighting, $\\lambda$ for knowledge distillation) whose optimal settings can be highly sensitive to the dataset and task sequence, and the paper offers no adaptive mechanism for tuning them. \n3.The reliance on a trusted third party for model collection and computation is still a form of centralization with potential privacy risks.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths\nThe work tackles a significant and difficult problem in FCL concerning dynamic heterogeneity. The core idea of using game-theoretic dynamic coalitions is novel and represents a solid contribution. Furthermore, the \"overall similarity\" proxy is a clever method to bypass the prohibitive computational cost of evaluating all possible coalitions directly. The experimental validation is also a major strength, featuring a comprehensive set of baselines, well-chosen datasets, and convincing ablation and communication cost analyses.\n\nWeaknesses\n1.The primary weakness is the method's scalability. The core algorithm's $O(K2^K)$ complexity is a fatal flaw for real-world federated learning with a large number of clients (K), a point the paper largely glosses over.\n2.The benefit function is a pure heuristic with no analysis of its failure modes; it's unclear when the $\\cos(g) + \\epsilon \\cdot \\cos(\\theta)$ proxy might be misleading.\n3.The paper lacks an overall convergence analysis. While the merge-blocking algorithm finds a static equilibrium for a given round, there's no proof that the entire FCL process, with dynamically evolving coalitions, converges to a meaningful solution.\n4.The decision to reform coalitions every single round seems inefficient and lacks justification compared to less frequent, event-triggered updates.", "was_revised": false, "final_justification": "", "rebuttal": "## Reviewer ckij\nWe would like to thank the reviewer for the positive and insightful comments. Below are our responses to the comments in **Weaknesses and Questions**.\n\n---\n\n#### ***Weakness 1& Question 1\n\n**Answer:** We thank the reviewer for their suggestions. While our current experiments focus on moderate-scale federations to study fundamental collaboration dynamics, we acknowledge the combinatorial complexity challenge. Though computational cost isn't our primary focus given our goal of precise collaborator identification for personalized clients, we recognize scalability limitations commonly in methods like [1]'s loss distance, [2]'s FC-based, or [3]'s gradient-based approaches. This reflects a common challenge in group-wise federated systems where pairwise similarity computations are unavoidable.\n\nTo address scalability we plan to use:\n\n1. **Sinkhorn Approximation**: For large client counts, we employ Sinkhorn's entropy-regularized optimal transport [4] to efficiently approximate similarity matrices through iterative refinement.\n\n2. **Localized Assessment**: Clients compute similarities only with Top-K neighbors [5] or historical collaborators, using compressed parameter representations/PEFT adapters for efficiency.\n\n3. **Sparse Matrix Construction**: We retain only high-similarity edges (above threshold) in the similarity graph.\n\n4. **Algorithmic Optimization**: Our merge-blocking algorithm processes large coalitions first and only traverses non-zero matrix entries, with BC-triggered merging/splitting per Algorithm 1.\n\nIn summery, we will leave these promising extensions to future work and clarify this in revision.\n\n[1] M. Zhang, K. Sapra, S. Fidler, S. Yeung, and J. M. Alvarez, \"Personalized Federated Learning with First Order Model Optimization,\" in Proc. Int. Conf. Learn. Represent..\n\n[2] Wu, S., Jia, Y., Liu, B., Xiang, H., Xu, X., & Dou, W. (2025). PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers. .\n\n[3] Sattler, F., Müller, K.-R., and Samek, W. Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints. .\n\n[4] Sinkhorn, R. A relationship between arbitrary positive matrices and doubly stochastic matrices. The Annals of Mathematical Statistics, 35(2), 876-879, 1964.\n\n[5] S. Li, T. Zhou, X. Tian and D. Tao, \"Learning to Collaborate in Decentralized Learning of Personalized Models,\" 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition .\n\n---\n\n#### ***Weakness 2& Question 2: \n\n**Answer:** Thank you for pointing that which inspires our consideration. 1.Nonlinear Separability: When clients share high feature similarity but opposing labels, the proxy may misgroup them. While rare in practice, XOR-style distributions demonstrate this limitation. 2.Extreme Drift: Abrupt task shifts (e.g., CIFAR-10→MNIST vs EMNIST→CT-RATE) can make knowledge transfer harmful, favoring local training over similarity-based aggregation.\n\n\n---\n\n#### ***Weakness 3& Question 3***: \n\n**Answer:** Thank you for your sincere remind. We analyze the convergence properties of our method from two perspectives:\n\n1. **Optimization Convergence at Client Level**:\nproblem setting\n\\begin{equation}\n\\min_{\\theta_k}L_k^t(\\theta_k)=L_{\\mathrm{class}}^t(\\theta_k)+\\lambda L_{\\mathrm{dis}}^t(\\theta_k)\n\\end{equation}\nclient update:\n\\begin{equation}\n\\theta_{k}^{\\tau+1}=\\sum_{i\\in S}\\alpha_{i}(\\theta_{i}^{\\tau}+\\Delta\\theta_{i}^{\\tau})\n\\end{equation}\nassumptions:\n1.L-smoothness\n\\begin{equation}\nL_k^t(y) \\leq L_k^t(x) + \\langle \\nabla L_k^t(x), y - x \\rangle + \\frac{L}{2} \\|y - x\\|^2\n\\end{equation}\n\n2.bounded variance\n\n\\begin{equation}\n\\mathbb{E}_\\xi \\left[\\|\\nabla L_k^t(x, \\xi) - \\nabla L_k^t(x)\\|^2\\right] \\leq \\sigma^2\n\\end{equation}\n\n3.unbiased estimation\n\n\\begin{equation}\n\\mathbb{E}_\\xi \\left[\\nabla L_k^t(x, \\xi)\\right] = \\nabla L_k^t(x)\n\\end{equation}\n\nproof:\n\n\\begin{align}\n\\theta_k^{{\\tau}+1} &= \\sum_{i\\in S} \\alpha_i (\\theta_i^{\\tau} + \\Delta\\theta_i^{\\tau}), \\quad \\Delta\\theta_i^{\\tau} = -\\eta \\nabla_\\theta L_k^{\\tau} (D_k^{\\tau}, \\theta_k^{\\tau}) \\\\\\\\\n&= \\alpha_k (\\theta_k^\\tau + \\Delta\\theta_k^\\tau) + \\sum_{i\\in S} \\alpha_i (\\theta_i^\\tau + \\Delta\\theta_i^\\tau), \\quad \\sum_{i\\in S} \\alpha_i = 1 \\\\\\\\\n&= \\theta_k^\\tau + \\Delta\\theta_k^\\tau - \\sum_{i\\in S \\setminus \\{k\\}} \\alpha_i (\\theta_k^\\tau + \\Delta\\theta_k^\\tau) + \\sum_{i\\in S \\setminus \\{k\\}} \\alpha_i (\\theta_i^\\tau + \\Delta\\theta_i^\\tau) \\\\\\\\\n&= \\theta_k^\\tau + \\Delta\\theta_k^\\tau + \\sum_{i\\in S \\setminus \\{k\\}} \\alpha_i [\\theta_i^\\tau - \\theta_k^\\tau + \\Delta\\theta_i^\\tau - \\Delta\\theta_k^\\tau] \\\\\\\\\n&= \\theta_k^\\tau - \\eta \\nabla_\\theta L_k^\\tau (D_k^\\tau, \\theta_k^\\tau) \\\\\\\\\n& + \\sum_{i\\in S \\setminus \\{k\\}} \\alpha_i [\\theta_i^\\tau - \\theta_k^\\tau - \\eta (\\nabla_\\theta L_i^\\tau (D_i^\\tau, \\theta_i^\\tau) - \\nabla_\\theta L_k^\\tau (D_k^\\tau, \\theta_k^\\tau))]\n\\end{align}\ndefine the disturbance term:\n\\begin{equation}\n\\delta_k^\\tau = \\sum_{i\\in S \\setminus \\{k\\}} \\alpha_i [\\theta_i^\\tau - \\theta_k^\\tau - \\eta (\\nabla_\\theta L_i^\\tau (D_i^\\tau, \\theta_i^\\tau)) - \\nabla_\\theta L_k^\\tau (D_k^\\tau, \\theta_k^\\tau)]\n\\end{equation}\nnorm upper bound of disturbance term:\n\\begin{equation}\n\\| \\delta_k^\\tau \\| \\leq \\epsilon := \\sup \\| \\delta_k^\\tau \\|\n\\end{equation}\nwhere\n\\begin{align}\n\\epsilon_{\\text{model}} &= \\left\\| \\sum_{i\\in S \\setminus \\{k\\}} \\alpha_i (\\theta_i^\\tau - \\theta_k^\\tau) \\right\\| \\\\\\\\\n\\epsilon_{\\text{grad}} &= \\eta \\left\\| \\sum_{i\\in S \\setminus \\{k\\}} \\alpha_i \\left(\\nabla_\\theta L_i^\\tau (D_i^\\tau, \\theta_i^\\tau) - \\nabla_\\theta L_k^\\tau (D_k^\\tau, \\theta_k^\\tau)\\right) \\right\\| \\\\\\\\\n\\text{satisfying} & \\quad \\| \\delta_k^\\tau \\| \\leq \\epsilon_{\\text{model}} + \\epsilon_{\\text{grad}} = \\epsilon\n\\end{align}\nIn our dynamic cooperation, $S$ differs with times.\n\\begin{align}\n\\theta_k^{\\tau+1} - \\theta_k^\\tau &= -\\eta \\nabla L_k^\\tau(\\theta_k^\\tau) + \\delta_k^\\tau \\\\\\\\\nL_k(\\theta_k^{\\tau+1}) &\\leq L_k(\\theta_k^\\tau) + \\langle \\nabla L_k(\\theta_k^\\tau), -\\eta \\nabla L_k(\\theta_k^\\tau; \\zeta_k^\\tau) + \\delta_k^\\tau \\rangle \\\\\\\\\n&\\quad + \\frac{L}{2} \\| -\\eta \\nabla L_k(\\theta_k^\\tau, \\zeta_k^\\tau) + \\delta_k^\\tau \\|^2 \\\\\\\\\n\\mathbb{E}[L_k(\\theta_k^{\\tau+1})] &\\leq \\mathbb{E}[L_k(\\theta_k^\\tau)] - \\eta \\mathbb{E}[\\|\\nabla L_k(\\theta_k^\\tau)\\|^2] + \\mathbb{E}[\\langle \\nabla L_k(\\theta_k^\\tau), \\delta_k^\\tau \\rangle] \\\\\\\\\n&\\quad + \\frac{L\\eta^2}{2} \\mathbb{E}[\\|\\nabla L_k(\\theta_k^\\tau, \\zeta_k^\\tau)\\|^2] + \\frac{L}{2} \\mathbb{E}[\\|\\delta_k^\\tau\\|^2] \\\\\n&\\quad + L\\eta \\mathbb{E}[\\langle \\nabla L_k(\\theta_k^\\tau, \\zeta_k^\\tau), \\delta_k^\\tau \\rangle]\n\\end{align}\nwith bounded conditions:\n\\begin{align}\n\\mathbb{E}[\\|\\nabla L_k(\\theta_k^\\tau, \\zeta_k^\\tau)\\|^2] &\\leq \\|\\nabla L_k(\\theta_k^\\tau)\\|^2 + \\sigma^2 \\\\\\\\\n\\|\\delta_k^\\tau\\|^2 &\\leq \\epsilon^2\n\\end{align}\nwe have:\n\\begin{align}\n\\mathbb{E}[L_k(\\theta_k^{\\tau+1})] &\\leq \\mathbb{E}[L_k(\\theta_k^\\tau)] \\\\\\\\\n&- (\\eta - C_1\\eta^2 - C_2\\eta \\epsilon - C_3 \\epsilon) \\mathbb{E}[\\|\\nabla L_k(\\theta_k^\\tau)\\|^2] \\\\\\\\\n&+ C_4 (\\eta^2\\sigma^2 + \\epsilon^2)\n\\end{align}\nset:\n\\begin{align}\n\\eta &= \\frac{1}{\\sqrt{\\tau}} \\quad \\Delta := L_k(\\theta_k^0) - L_k^* \\leq C \\\\\\\\\n\\gamma &:= \\eta - C_1\\eta^2 - C_2\\eta \\epsilon - C_3 \\epsilon \\\\\\\\\n\\tau &= 0 \\cdots T-1 \\\\\\\\\n\\mathbb{E}[L_k(\\theta_k^{\\tau})] &\\leq \\mathbb{E}[L_k(\\theta_k^0)] - \\sum_{\\tau=0}^{T-1} \\gamma \\mathbb{E}[\\|\\nabla L_k(\\theta_k^\\tau)\\|^2] \\\\\\\\\n&+ TC_4 (\\eta^2\\sigma^2 + \\epsilon^2) \\\\\\\\\n\\sum_{\\tau=0}^{T-1} \\mathbb{E}[\\|\\nabla L_k(\\theta_k^\\tau)\\|^2] &\\leq \\frac{L_k(\\theta_k^0) - L_k(\\theta_k^{\\tau})}{\\gamma} + \\frac{TC_4}{\\gamma} (\\eta^2\\sigma^2 + \\epsilon^2) \\\\\\\\\n\\frac{1}{T} \\sum_{\\tau=0}^{T-1} \\mathbb{E}[\\|\\nabla L_k(\\theta_k^\\tau)\\|^2] &\\leq O\\left(\\frac{1+\\sigma^{2}+\\epsilon^{2}}{\\sqrt{T}}\\right)\n\\end{align}\nwhich indicates optimization convergence on each client.\n\n2. **Coalition evolution convergence**: Our merge-blocking strategy ensures that the coalition structure evolves toward a stable state.\n\n---\n\n#### ***Weakness 4& Question 4: \n\n**Answer:** Our framework design explicitly addresses this concern through experimental validation of coalition update frequencies. Results across 10-communication-round task phases demonstrate that per-round coalition evaluation, while computationally intensive, is essential for handling unpredictable data arrival patterns during task learning.\n\n| Update Times | 1 | 1 | 2|2|  5| 5| 10|10|\n|--------------|---------------|---------------|---------------|----------------|---------------|---------------|---------------|----------------|\n|| Accuracy   | Forget | Accuracy  | Forget | Accuracy   | Forget | Accuracy    | Forget |\n| EMNIST-LTP         | 42.4 | 15.6   | 49.3 | 13.1   | 50.6 | 10.4   | **52.5**  | **8.9**    |\n| EMNIST-shuffle   | 70.9 | 12.9   | 71.5 | 12.3   | 74.3 | 9.6    | **78.3**  | **4.2**    |\n\nTimely collaborator identification is critical as mini-batches arrive during task training. Infrequent coalition updates risk failing to adapt to emerging patterns - e.g., when learning new classes like \"dog\", delayed updates maintain outdated collaborations, increasing catastrophic forgetting and accuracy degradation.\n\n---\n\n#### ***Limitations 2:***\n**Answer:** Thank you. We acknowledge this parameter sensitivity and plan to implement auto-tuning in future work.\n\n---\n\n#### ***Limitations 3***\n\n**Answer:** Thank you. In practice, an impartial third-party (e.g., industry association) handles model collection, benefit assessment, and cooperation publishing in our federated learning paradigm. Crucially, our approach differs from centralized learning in two aspects: Only evaluates optimal coalitions (preventing self-interest) without performing model aggregation/distribution, which remains client-autonomous. Preserves client-specific learning trajectories via decentralized knowledge transfer, avoiding global model imposition.\n\nWe appreciate the privacy concerns raised and will address trusted third-party aspects in future work."}, {"review_id": "VJs2PrPsxp", "reviewer": "Reviewer_1yUZ", "summary": "This paper proposes a decentralized dynamic cooperation framework where clients form adaptive coalitions for personalized learning, leveraging affinity games and new algorithms to achieve superior performance over existing baselines. The proposed framework is partially validated on small-scale experiments, without providing relevant theoretical guarantees.", "questions": "(a) Is the concept of decentralization introduced in this paper merely an arbitrary grouping of clients? This does not seem to capture the true essence of decentralization. Although some explorations of push-sum-like topologies have suggested that dynamic links might offer better alternatives, the analysis of such schemes is still fundamentally based on the investigation of consensus constraints or spectral gap properties. It appears that the present paper has misused the concept of decentralization. The authors are requested to clarify the similarities and differences between their approach and traditional decentralization. In my opinion, these concepts should not be conflated, and the paper seems to require substantial revision.\n\n(b) What is the overall optimization objective of this paper? At present, it seems that only the local optimization form on the client side has been defined, which appears to be completely unrelated to personalized federated learning. I would like to know the specific form of the global optimization objective.\n\n(c) The content in Section 3.3 does not appear to be original. The authors should cite relevant literature and clearly indicate which parts of this section are their own contributions.\n\n(d) Please provide the computational complexity or training wall-time curves for both the baseline methods and the proposed method. The approach presented in this paper involves optimizing with four loss functions, which results in significantly higher computational complexity compared to the original methods. However, the paper does not report any metrics related to training cost. Based on the current description, the proposed method may be extremely expensive in terms of computational resources.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\nThe paper provides a detailed introduction to the proposed dynamic collaboration process, offering a reference model for the federated continual learning framework. It also incorporates grouping schemes based on similarities in gradients and model parameters, providing practical guidance for engineering applications.\n\nWeaknesses:\n\nThe classification of the basic settings of federated learning in this paper is incorrect. For example, in the paragraph around line 39, when discussing performance comparisons under different settings, the paper conflates general FL, PFL, and even DFL. The goal of general FL is to collaboratively train a single global model across multiple nodes, while the objective of PFL is to train multiple local models. DFL refers to a communication method induced by different topology; it can be used to accomplish both FL and PFL training. I suggest the authors revise the entire manuscript, or at least clarify the classification and task objectives related to FL throughout the paper.\n\nThe motivation of this paper is not clearly stated. Simply adopting aggregation to avoid catastrophic forgetting, or using decentralized aggregation to improve the performance of models within groups, makes the work appear incremental. It is currently unclear what core challenge the submission aims to address. Could the authors further clarify, perhaps through mathematical formulations or other quantifiable characteristics, what specific problems this series of techniques is designed to solve?\n\nThe series of techniques proposed in this paper do not appear to be fully original, and they also lack theoretical analysis and guarantees.\n\nThe experimental setup is small and differs significantly from typical federated learning settings. Standard FL usually involves collaborative training with hundreds of clients, whereas the setup in this paper appears to be limited to only 8 to 10 clients. Thus it becomes much easier to achieve significant control through data partitioning or task allocation. However, in practice, when hundreds of clients are involved in training, certain similarity measures and training methods may not be as effective as they are in small-scale experiments.", "was_revised": true, "final_justification": "The optimization problem in this paper is not well described. After multiple discussions with the authors, I have come to understand the actual optimization objective and the corresponding solution approach. The game-theoretic method for FL aggregation proposed in this work is not novel, but the current solution approach is correct. Its advantage lies in its ability to strictly guarantee balance on each communication round, while its drawback is that the computational complexity grows exponentially with the number of clients. At present, I have no further concerns and will raise my score to 4.", "rebuttal": "## Reviewer 1yUZ\nWe would like to thank the reviewer for the insightful comments. Below are our responses to the comments in **Weaknesses and Questions**.\n\n---\n\n**Respond to W1**\n\n**Answer:** We thank the reviewer for their suggestions. Personalized FCL enables clients to maintain individual task histories and learn customized models through cooperation [1,2]. Unlike centralized approaches, decentralized learning eliminates the need for a central server [2,3], better aligning with our goal of optimizing individual client performance (Eq.1) rather than global objectives. As discussed in Section 1 (Paragraphs 2-3), decentralized topologies are more suitable for personalized FCL. The comparison of CFL, Per-FedAvg, and centralized methods (Section 1, line 39) demonstrates that selective cooperative aggregation is essential. We apologize for any lack of clarity and will refine these definitions in the revised manuscript.\n\n---\n**Respond to W2**\n\n**Answer:** We thank the reviewer for arguing this point. Our work addresses the key challenge of ***how to achieve personalized continual learning in federated systems with strong spatio-temporal catastrophic forgetting*** as stated in Section 1. \n\n\n\n1. FCL Challenges\nFCL introduces two key challenges:\n(1) Local temporal catastrophic forgetting from sequential task learning\n(2) Spatial catastrophic forgetting during aggregation of heterogeneous models\n\n2. Assumptions and Initial Evidence\nWe first establish that federated aggregation can mitigate forgetting when models share homogeneous knowledge (Fig.1a-b), explaining central server architectures [4,5].\n\n3. Limitations of Complete Aggregation\nComplete aggregation proves suboptimal under spatial heterogeneity due to knowledge interference (Fig.1c), showing aggregation efficacy depends on methodology.\n\n4. Connection to Personalized FL\nWhile methods like ClusterFL (decentralized) and Per-FedAvg handle spatial heterogeneity, they fail to address continual learning challenges (lines 45-49).\n\nThis motivates our DCFCL framework featuring dynamic grouping to:Handle continuous task arrivals，Prevent spatio-temporal forgetting，Maintain decentralized personalization\n\n\n\n---\n\n**Respond to W3**\n\n**Answer:**We thank the reviewer for raising this point. We are the first to propose a dynamic grouping strategy for personalized model learning in FCL which is entirely original. We would like to clarify our contributions as follows:\n1.**Theoretical Foundation**\nSection 3.1 formally establishes the problem of personalized FCL in decentralized topologies. Section 3.2 introduces our novel dynamic grouped aggregation approach for model updates. Section 3.3 applies cooperative game theory to determine optimal groupings, with the cooperative equilibrium as our target solution.\n\n2.**Core Innovations in Chapter 4**\nWe extend static cooperative games to dynamic settings to achieve dynamic grouping. While inspired by coalitional affinity games for relationship modeling, all algorithmic components are original:(1) Knowledge distillation (creatively applied) maintains feature consistency for cooperator identification\n(2) Our benefit table framework uses:\nOriginal overall similarity metric,Coalitional affinity principles\n(3) We created:Merge-blocking algorithm,Dynamic cooperative evolution algorithm,to obtain Dynamic Cooperative Equilibrium\n\n3.**Theoretical Analysis**\nAppendix A contains our original arguments about coalitional affinity game applications. Extensive experiments validate:KD,OS,CE.All aspects from problem formulation to equilibrium analysis represent original FCL contributions, except: affinity-based relationship concept,KD's core mechanism.\nWe believes that this can serve as theoretical analysis and guarantee to a certain extent. If we have any misunderstandings, please do not hesitate to question us. We will do our best to clear up your doubts.\n\n---\n\n**Respond to W4**\n\n**Answer:** Our experimental results (Fig.7, Appendix D.4) demonstrate that while conventional equilibrium computation exhibits exponential time growth as client numbers increase from 10 to 50, DCFCL maintains near-linear scalability - a performance gap that widens significantly with system scale. The proposed Merge-Blocking Algorithm achieves substantial efficiency gains, including faster equilibrium formation and fewer iterations, while significantly reducing both time complexity  and communication costs  compared to baseline methods.\n\n|Number of clients|10|20|30|40|50|\n|-|-|-|-|-|-|\n|Time (min)|0.116|2.715|8.279|12.526|2.715|\n|Iterations|2|2|3|3|5|\n|Time (min)-w/o MBA|1.438|8.504|24.733|39.293|54.596|\n\nWe acknowledge the combinatorial complexity challenge, though our current focus remains on moderate-scale federations to study fundamental collaboration dynamics. While computational cost isn't our primary focus given the priority of precise collaborator identification, our framework demonstrates strong performance/speed trade-offs versus alternatives ([6]'s loss distance, [7]'s FC-based, [8]'s gradient-based). Like most group-based FL methods, pairwise similarity computations inherently limit scalability for very large systems. To address scalability we plan to use:1.Sinkhorn Approximation: For large client counts, we employ Sinkhorn's entropy-regularized optimal transport [9] to efficiently approximate similarity matrices through iterative refinement. 2.Localized Assessment: Clients compute similarities only with Top-K neighbors [10] or historical collaborators.3.Sparse Matrix Construction: We retain only high-similarity edges in the similarity graph.4.Algorithmic Optimization: Our merge-blocking algorithm processes large coalitions first and only traverses non-zero matrix entries.\n\nIn summery, we will leave these promising extensions to future work and clarify this in revision.\n\n---\n\n**Respond to Q1**\n\n**Answer:** We clarify our decentralized FCL approach through four key aspects:\n\nDecentralization Advantage: Unlike centralized aggregation, our decentralized topology better enables personalized model formation, achieving client-satisfactory group aggregation without central server involvement.\n\nCooperative Game Foundation: Our dynamic group strategy is theoretically grounded in cooperative games, where rational clients form performance-optimizing coalitions while excluding non-contributors - fundamentally differing from clustering methods.\n\nBehavioral Personalization: Client autonomy in alliance formation/betrayal embodies personalization, with the decentralized structure naturally aligning with cooperative game theory's agent-centric dynamics.\n\nTraditional Comparison: While traditional decentralized methods ([11]'s gossip-SGD, [12]'s doubly-stochastic weights) enforce global consensus ([13]), they falter in non-IID settings ([14]). Like [15,16], we adopt decentralization for personalized modeling, but uniquely integrate it with continual learning.\n\n---\n\n**Respond to Q2**\n\n**Answer:** Thank you for your questioning. Our overall optimization goal is specific to each client, which is precisely what personalized FL defines[17,18], as described in Section 3.1 Problem Setup. At each task phase $t$, model parameter of client $k$ is $θ_k^t$ , and optimization goal is foreach client. Because it is personalized model and adopts a decentralized structure, we do not have a global optimization objective. This is the main difference from the centralized method, as the latter may optimize a global objective.\n\n---\n\n**Respond to Q3**\n\n**Answer:** Thank you for your questioning. In the revised version, we will cite the article of cooperative game theory.\n\n---\n\n**Respond to Q4**\n\n**Answer:** We provide comprehensive runtime comparisons with all baselines across datasets in Appendix D.4 (full training wall-time), along with time complexity analysis versus non-optimized cooperative strategies in Appendix B.4. Our method avoids four-loss optimization, and we detail communication costs (Appendix D.2) and computation costs (Appendix D.4). We apologize for any unclear presentation and will clarify these analyses in revision.\n\n\n\n[1] J. Xie, FedMeS: Personalized Federated Continual Learning Leveraging Local Memory.\n\n[2] S. Li, Learning to Collaborate in Decentralized Learning of Personalized Models,.\n\n[3] S. Kharrat, DPFL: Decentralized Personalized Federated Learning.\n\n[4] Yoon, J. Federated continual learning with weighted inter-client transfer.\n\n[5] Shenaj, D.. Asynchronous federated continual learning.\n\n[6] M. Zhang, \"Personalized Federated Learning with First Order Model Optimization.\n\n[7] Wu, S.. PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers.\n\n[8] Sattler, F.. Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints..\n\n[9] Sinkhorn, R. A relationship between arbitrary positive matrices and doubly stochastic matrices. \n\n[10] S. Li, T. Zhou, X. Tian and D. Tao, \"Learning to Collaborate in Decentralized Learning of Personalized Models.\n\n[11] Michael Blot, . Gossip training for deep learning. \n\n[12] Zhanhong . Collaborative deep learning in fixed topology networks. \n\n[13] Xiangru Lian, . Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent.\n\n[14] Kevin Hsieh, The non-iid data quagmire of decentralized machine learning.\n\n[15] S. Li, \"Learning to Collaborate in Decentralized Learning of Personalized Models.\n\n[16] P. Vanhaesebrouck, , \"Decentralized Collaborative Learning of Personalized Models over Networks,\" arXiv preprint arXiv:1610.05202, 2017.\n\n[17] S. Li, \"Learning to Collaborate in Decentralized Learning of Personalized Models,\" 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition.\n\n[18] Wu, S., . PFedCS: A Personalized Federated Learning Method for Enhancing Collaboration among Similar Classifiers. ."}]

Author rebuttal:

Meta-review: This paper introduced a novel decentralized federated continual learning framework. This framework enforced the dynamic cooperation among clients to mitigate catastrophic forgetting and improve model performance.

Strengths:
- The paper is well-motivated.
- The proposed similarity metric for FCL is both theoretically grounded and computationally efficient
- The method is well validated across various benchmarks, followed by detailed ablation studies and parameter sensitivity analyses.

Weakness:
- The conceptual distinction between decentralized FCL and personalized FCL can be further clarified to highlight the FCL problem studied in this work. 
- The space complexity analysis can be clarified.
- More insights can be provided to explain the impact of hyperparameters in the experiments.
- The scalability of the proposed method can be analyzed and improved.
- The notation and formulation used in this paper can be further clarified.

Recommendation

Though there are some concerns related to clarity and complexity analysis, all reviewers provided positive feedback overall. I recommend acceptance, and encourage the authors to address the reviewers’ suggestions to further improve the quality and clarity of the paper.
Decision: Accept (poster)