Federated Few-Shot Class-Incremental Learning
Authors: Muhammad Anwar Ma'sum
Mahardhika Pratama
Lin Liu
H Habibullah
Ryszard Kowalczyk
Venue: ICLR 2025 Poster (2025)
Keywords: Federated
Few-Shot
Class-Incremental Learning
Prototype-bias
Rectification
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: This study proposes a challenging yet practical Federated Few-Shot Class-Incremental Learning (FFSCIL) problem, where clients only hold very few samples for new classes.  We develop a novel Unified Optimized Prototype Prompt (UOPP) model to simultaneously handle catastrophic forgetting, over-fitting, and prototype bias in FFSCIL. UOPP utilizes task-wise prompt learning to mitigate task interference and over-fitting, unified static-dynamic prototypes to achieve a stability-plasticity balance, and adaptive dual heads for enhanced inferences. Dynamic prototypes represent new classes in the current few-shot task and are rectified to deal with prototype bias. Our comprehensive experimental results show that UOPP significantly outperforms state-of-the-art (SOTA) methods on three datasets with improvements up to 76% on average accuracy and 90% on harmonic mean accuracy respectively. Our extensive analysis shows UOPP robustness in various numbers of local clients and global rounds, low communication costs, and moderate running time. The source code of UOPP is publicly available at https://github.com/anwarmaxsum/FFSCIL.

Published as a conference paper at ICLR 2025
FEDERATED LEARNING
FEW-SHOT
CLASS-INCREMENTAL
M. Anwar Ma'sum", Mahardhika Pratama, Lin Liu, Habibullah Habibullah, and Ryszard Kowalczyk University of South Australia, Mawson Lakes, SA, 5095, Australia
masmy039 @ mnymail.unisa eni.a. dhika. wn...e.eu.uuuuuuu...... lin. @ @ww.ia.nnuu....m habibullah. habibullah @
ryszard.kowalczykke udisa.edu.au
ABSTRACT
This study proposes challenging yet practical Federated Few-Shot Class- Incremental Learning (FFSCIL) problem, where clients only hold very few sam- ples for new classes. We develop novel Unified Optimized Prototype Prompt (UOPP) model to simultaneously handle catastrophic forgetting, over-fitting, and prototype bias in FFSCIL. UOPP utilizes task-wise prompt learning to mitigate task interference and over-fitting, unified static-dynamic prototypes achieve cubbilitypllasticonn balance, and adaptive dual heads for enhanced inferences. Dy- namic prototypes represent new classes in the current few-shot task and are rec- tified to deal with prototype bias. Our comprehensive experimental results show that UOPP significantly outperforms state-of-the-art (SOTA) methods three datasets with improvements up to 76% on average accuracy and 90% on harmonic mean accuracy respectively. Our extensive analysis shows UOPP robustness various numbers of local clients and global rounds, low communication costs, and moderate running time. The source code of UOPP is publicly available https: / / thub. com anwa rma www.asumm/aaammmmmmmmmmm MKssu/FFFSC... FFSCIL.
INTRODUCTION
The previous studies Federated Class Incremental Learning (FCIL) address catastrophic forgetting chal- lenges in dynamic environment with data privacy constraints. Coor- dinated by central server, a collec- tion of clients continually develops global recognition model without sharing their local data. The first is- sue of FCIL is the existing works i.c. LGA (Dong et al., 2023), TARGET (Zhang et al., 2023b) and LANDER data and thus impractical in the resource-comee constrained environments. They are data-hungry such that they face the issues of prototype bias and over-fitting in realm of the data scarcity constraint. As in stand-alone few-shot learning (Zhang et al., 2022), federated learning with few samples leads to prototype bias problems. Figure 1 visualizes how prototypes of the observed classes are generated and aggregated from the few samples carried by the clients. In FCIL simulation (Dong et al., 2023; 2022), where client holds only partial classes e.g. 60% of the total classes, shows that the ob- served few samples by client lead to prototype bias where prototype doesn't 't represent the true population rather it represents the few locally gathered samples. To overcome the prototype bias problem, each prototype should be refined to correct location. As shown in Figure 1, we also emphasize that aggregating the prototype can't handle the prototype bias. Few-shot learning (FSL) e.g. MetaNode (Zhang et al., 2022) or few-shot class incremental learning (FSCIL) e.g. S3C (Kalla & Biswas, 2022) methods can't 't be expected since the methods require the presence of all classes whereas, in federated setting, client carries only a subset of all classes.
ON iosall
By ster whoc cculkuky
Id aller wit Belleengg
ger cirn
Ssarpe Cilommaaeee
Coservation
The importance of prototype rectification to handle pro- totype bias (a) Initial prototype per client (b) Aggregation without rectification can't handle prototype-bias (c) Aggregation with recti- fication overcomes prototype-bias
Figure 1:
(Tran et al., 2024) assume the clients carry abundant training
"Corresponding author

---

Published as a conference paper at ICLR 2025
Second, the current FCIL methods train and share whole backbone parameters resulting in large number of parameters during optimization processes which imply a long training time and a high communication cost. Third, in the current SOTAs, clients generate and share synthetic images for global model aggregation on the server side. Aside from overloading the communication costs, this mechanism may violate data privacy principles since synthetic data may reveal partial information about the private data of a client to another party due to its similarity to the original data. Last but not least, some FCIL methods e.g. LGA (Dong al., 2023) GLFC (Dong et al., 2022) save several exemplars from previous tasks for rehearsal that may breach data openness policy, where data are only open for a client at a specific moment.
These gaps motivate us to address new direction of FCIL Federated Few-Shot Class Incremen- tal Learning (FFSCIL) where a client participating in the federated learning process only possesses very few samples. Second, we develop a novel efficient but effective approach to the FFSCIL prob- lem, where a client trains and shares as small parameters as possible but produces a highly accurate global model without sharing any synthetic samples or saving exemplars from previous samples. Therefore, our proposed method handles catastrophic forgetting in dynamic collaborative learning with data privacy and data scarcity constraints. The contributions of this paper are: (1) We empha- size the data scarcity issue leading to the prototype-bie bias and over-fitting problems in FCIL and define new problem, namely Federated Few-Shot Class Incremental Learning (FFSCIL); (2) We propose a novel method for the FFSCIL problem termed Unified Optimized Prototype Prompt (UOPP) built upon prompt learning framework coupled with static and dynamic prototypes optimized by Neu- ral Ordinary Differential Equation (ODE). The proposed method utilizes an adaptive dual-head to enhance its predictive accuracy. To our knowledge, our proposed method is the first prompt-based that integrates prompt tuning, prototype rectification by trainable network and adaptive dual clas- sifiers in single pipeline; (3) We offer theoretical studies for the convergence and generalization of the proposed method; (4) We provide a comprehensive analysis in three benchmark datasets that show the proposed method outperforms the baseline and current SOTAs with significant gaps and achieves improved stability-plastier balances. Our analysis emphasizes the robustness of the pro- posed method in various participating clients and small rounds per task.
2 RELATED WORKS
Federated Class Incremental Learning (FCIL): The FCIL studies address catastrophic forgetting problem while preserving dta privacy e.g. FedWeIT (Yoon et al., 2021), GLFC (Dong al., 2022), and LGA (Dong et al., 2023) optimize the global model by aggregating locally optimized models by the participating clients. The current SOTAs prove their effectiveness rather than combining Fe- d'vg (McMahan et al., 2017) and class incremental learning (CIL) method such as ICARL(Rebuffi et al., 2017) and BiC (Wu et al., 2019). However, the SOTAs tune and send the whole backbone, producing long training times and high communication costs as a consequence. Furthermore, the SOTAS assume that a client saves several samples as memory that may not be practically applicable. Other studies i.e. TARGET (Zhang et al., 2023a) leverage synthetic samples for rehearsal instead of real samples. achieves higher performance than FedWeIT (Yoon et al., 2021) but still outper- formed by LGA. A different approach i.e. FedCIL (Qi et al., 2023) trains generative model i.c. ACGAN (Odena et al., 2017) on local clients side to generate fake samples for aggregation on the central server's side. It achieves higher performance than the combination of FedAvg FedProx (Li et al., 2020) with ACGAN, DGR(Shin et al., 2017) or LWF-2T (Usmanova et al., 2021), but it needs more expensive communication costs and training time due to the generative model.
Few Shot Class Incremental Learning (FSCIL): Previous studies on FSCIL have attempted to maintain stability-plasticity tradeoff under data scarcity by adding extra representation e.g. TOPIC (Tao et al., 2020b) introduces Neural gas as the graph of mapped features and CEC (Zhang et al., 2021) continually evolves its classifier to adapt to new tasks. Another approach modifies its learning mechanism e.g. FSLL (Mazumder et al., 2021) updates with self-supervised loss, F2M (Shi et al., 2021) finds flat minima regions on the base task then forces parameter updates on few shot tasks to reside within the flat region, S3C (Kalla & Biswas, 2022) trains scholastic classifier with supervised loss and MgSvF (Zhao et al., 2024) applies multi grained fast-slow learning mechanism. Prototype- based methods e.g. TEEN et al., 2024), NC-FSCIL et al., 2023), and OrCO(Ahmed et al., 2024) show the important of prototype correction to deal with prototype bias in FSCIL. How- ever, prototype refinement in the data scarcity is still open challenge. Besides, FSCIL methods aren't 't yet proven in federated settings under non-i.i.d constrain. A comprehensive literature review is presented in Appendix G.
2

---

Published as a conference paper at ICLR 2025
+ Class
Unification
2
Prototype Rectricyttnn
Dual Head
Clay
Tuaeute chundll
Unficationn
Dual Hoad (Clasatfier
Prolatypa Recilfcation
HUP Hood
+
bosed Head
Pieteype
2
inthenctio
(b)
Detailed Unification, Rectification and Dual Head modules
von vii Erecder
Adcilinaa Poos Xeher Camee
VIT Encoder
MLP
Clkent,
Central Server
Task-O
Clooows
MLP Handd
+
(Prosdopp- pa- prsted Hoadt
Z;
(AG..P)
2:
ASAA U
Dog
Unitication
Prototype Reciifiction
KF
(Ai cartolgpting dieents)
Qu
Ray
Viu
Prompt
Frog
VIT Backbbone
(Frazen)
MBA
(K)
Pt)
P/
(K)).)))
Client,
Dual Head
W
Vesigttd Augrregation
Closons
MiP Hendi
pasad Piedd
Zi
Norm
Horze
Unification
Protolypa Rectifiatiio
(K Pi)
Z 6,
Taskit
T
Dog
X2
Entoedded Patchaa
Lion
VIT Backboon
(Frozen)
MSA
(AC
Pt)
12
Zi carine ausiittle cistse pobsoper: dog
Boo
(a) Local training at client
(c) Federated training, interaction between clients and central server
Figure 2: The visualization of UOPP, includes task-wise prompt learning empowered by shared unified static- dynamic prototypes, dynamic prototypes rectification, adaptive dual heads, and weighted aggregation. The gray-colored parameters are frozen and unshareable parameters, the green-colored parameters are trainable and shareable parameters, while the blue-colored parameters are trainable but unshareable parameters
PROBLEM FORMULATION
Federated Few-Shot Class-Ineremental Learning (FFSCIL) defined as: Given a sequence of tasks [0, 2.., T] where each task L carries labeled training set T {(a),, x()) i=1 where xf E X denotes an imput image and y E Y denotes its label, and ).| denotes the cardinality. Each task & is disjoint with another task t'. i.e. Ve, nT = O. On each task-t, set of clients (1) fau coordinated by central server G are deployed to learn T In the first task (1 0), each client I carries abundant training samples while the remaining tasks (t >> 0), it carries far smaller samples than the first task i.e 17, >> (T,12)). For convenient way, task-0 is called the base task while the rest is called the few-shot task (FS task). A client / holds only subset of current task training set i.e. T C CT, Each client-l carries non-identically and distributed data (non-i.i.d) to another client I' i.c. Vt # Dj, where D/ and Dj, are distribution of Ti' and T/ respectively. Following FCIL, Non-i.i.d distribution is represented by the percentage of available classes 17. Each task-t is learned in federated way that is repeated in Ry rounds where in each round r [1..Rr]. a set of local clients is randomly selected from all available clients i.e. (1) C (1)=r Juu Due data privacy constraints, a client-/ is not allowed to share any training sample (I;, Mk) E T to another client or server, but permitted to share its parameters.
Let a deep neural network go (fe(.)) be parameterized by θ and & where f(.) and g(-) are the feature extractor and classifier respectively. In each round of task 6, central server G coordinates selected local clients {1)/11 to conduct local CIL training using its training samples [TT). Each client-/ optimizes its local parameters (0,, 4)). then sends its locally optimized parameters to the central server G' to be aggregated. The central server G aggregates all received local parameters into optimum global parameters i.e. (OG, &G) = Agg( ((((( (/)))= and communicates them back to all clients for the next round process. The objective of FFSCIL is to achieve an optimum global model a (fag()) to recognize the learned classes from the first task (task-0) until the current task (lask-t) i.c. (7 "..,, T'}.
4 PROPOSED METHOD: UNIFIED OPTIMIZED PROTOTYPE PROMPT(UOPP)
We design our method, termed Unified Optimized Prototype Prompt (UOPP), to address challenges in FFSCIL i.c. catastrophic forgetting, over-fitting and prototype bias simultaneously under data privacy constraints. Figure 2 exhibits the flow of our method in both the local training view (a) and federated training view (b). Looking at Figure 2 (a), we utilize a prompt-based approach on top of the frozen ViT backbone as it minimizes task interference that leads to better handling of catastrophic forgetting with lightweight learnable parameters (prompts). Then we add a rectification block to handle prototype bias in the fhw- tasks (t > 0) by iteratively rectifying the prototypes.
3

---

Published as a conference paper at ICLR 2025
Note that each client only holds few samples for any class in the few-shot tasks. Then add unification block to unify the rectified prototypes and the feature produced by the ViT encoder. Since the prototypes
shareable to/from the server, this mechanism handles non-iid challenge where a client-l is accommodated to learn knowledge representation from the classes not available in T Last, design an adaptive dual head that leverages the strength of both MLP and prototype-basee classifiers. On the federated view (b), the Figure shows that in our method, a client shares only small- sized parameters i.e. prompts, prototypes, and MLP parameters rather than the whole backbone parameters. Thus, it minimizes the communication cost between clients and the central server. The details of our method are presented in the following subsections. The uniqueness of our method from the existing prototype- based methods i.e. (Wang et al., 2024), (Yang et al., 2023), (Ahmed et al., 2024), (Goswami et al., 2024), and (Guo et al., 2024) that we utilize trainable Neural ODE that works by support and query samples drawn from different distribution, while the existing method utilizes similarity ratio for rectification process. Second, alongside prototype rectification, adjust task-wise key and prompt parameters to improve inter-task separability. Different from PILoRA (Guo et al., 2024), the prompt is prepended to W x and W x of VIT model, while PILoRA appends matrices A.B into and WV and doesn't 't use task keys. Third, we utilize dual-head classifiers to leverage the strength of both combined with task prediction.
4.1 MINIMIZING TASK INTERFERENCE VIA PROMPT LEARNING
Prompt Structure: define prompt P. for each client-l to Iearn sequence of task {t)
P PP), (K? Pf),.., (K?", Pf )..t t E [0..T] where (K₁, Pf) is a pair of prompt key-and- value corresponded to task-t. Our method utilizes the prefix tuning technique (Li & Liang, 2021) because it usually outperforms the prompt tuning method, then following the definition of prefix tuning, and attention mechanism in ViT, the output of ViT backbone given an input and prompt (K), Pf) is defined as:
((K),P4)()) = A(Qu, (K7 # KP + Visl)
where Qiy. Kag. Vis are query, key and value of jth head MSA in 7th layer of VIT encoder, # de- notes concatenation, and A denotes attention function. Note that the function is applied for all MSA heads j E [1..J] and all encoder layers i E [1..7]. At task-t, client-l optimizes only (K), Pt) and not the other prompt key-value pair, and (K(,P)) ) is adjusted only in task t, not in the previous or upcoming tasks. Therefore, once optimal to T, the pair (K), Pf) will remain robust against forget- ting since its value is not adjusted afterward. In addition, Ki is adjusted to match sample I; E T, by using a matching loss L. during training. This mechanism is designed to make (K) Pf) exclusive to task-t and not the samples from other tasks. In other words, it minimizes task interference. In addition, since only prompts rather than the whole network parameters are adjusted, this strategy alleviates the over-fitting problem due to few samples.
4.2 UNIFIED STATIC-DYNAMIC PROTOTYPE AND ITS USABILITY
Static Prototype: define a vector of D-dimension &. as the prototype for class T that is pro- duced from ViT encoder, where D is the embedding dimension. A static prototype set 2; { is a collection of static prototypes of class-c that are available at client-l. Assuming that prototype fol- lows a Gaussian distribution i.e. 2 N(pe Y.) and forms D disjoint uni-variate distribution, then a prototype of class e is represented as 2e ~ N(ue, a.?) where a.² = JD- 2 i E 21.2.... D). Id is identity matrix. Note that at task-t, selected local client-/ holds its local training set T, Suppose that The = {(af,. vit) E Tí' vie = c} is the samples of class-c in T, and |.| denotes the number of samples. Then, static prototype Esc ~ NN(pte o) for class-c available in T,' is computed by Eq. 2 and 3.
1 IT!
Mic
ITA
Mect,ep (15) x E Tie
(2)
1
(ui. fickepp(x.))) x E Tíe
(3)
IT
Dynamic Prototype: we define a vector & of D dimension as the prototype for class c E T iteratively rectified during local training task-t, where D is the embedding dimension. A dynamic prototype set 2, {âic} is collection of dynamic prototypes of class-c available at client-l, At the beginning of local training at task > on elient-l, a dynamic prototype &te is set by its corre- sponding static prototype value E.c. then iteratively rectified by GradNet go, during local training
4

---

Published as a conference paper at ICLR 2025
Le. Eee = Recti fication(st Zte). After finishing the training of task-t, the dynamic prototypes of all classes с E T are not updated anymore and are stored as static prototypes. Therefore, the dynamic prototype set 2 includes prototypes from currently learned classes only, while the static prototype set Z includes both currently learned classes and previously learned classes. The details of the rectification process are explained in the following sub-section.
we define the unified prototype Zi- Z U 2 as the union of static and dy- namic prototype of client-l. Following the definition of static and dynamic prototypes, Zi con- tains static prototypes of previously learned classes and dynamic prototypes of currently learned classes that are still updated during the local training at client-l. Unified prototype Zi plays an important role in our method, both in the base task (f = 0) and the few-shot task (t 0). As illustrated by the unification block in Figure 2, in the base task, Z is unified with the output of VIT i.e. [(Kt.rf)(x)) Since 7 is shareable, then Z contains prototypes of classes unavailable in Tt' shared by the central server. Therefore, each client-l can afford to learn all classes in T In the few-shot task, each client-l rectifies âcc for all classes E Tí. Similarly, the shareable Z enhances the separability of &te since it is contrasted to all other prototypes of all learned classes.
Unified Prototype:
4.3 HANDLING PROTOTYPE-BAAT BIAS VIA DYNAMIC PROTOTYPE RECTIFICATION
Previous studies (Chen et al., 2018; Zhang et al., 2022) emphasize that generations of prototypes by averaging few samples leads to prototype bias, as the prototypes only represent the observed samples but not the whole population, as illustrated in Figure 3(a). Therefore, our method rectifies prototype & to become as close as pos- sible to the population mean. We follow the framework
de
Phoobse
Photchpe
Figure 3: Visualization of Biased and Opti- mal Prototype (a), and Rectification (b)
of episodic training in few-shot learning (Vinyals et al., 2016; Zhang et al., 2022), a rectification pro- Rectificcaionn fication (gy O) &c ( Q, s = M) where s is the number of rectification steps, (0) and &c(M are the initial prototype and final prototype after M- -ssepps rectification process. S is the support set that contains few observed samples per class, Q is the query set comprising unlabeled samples, and go() is gradient networks (GradNet) parame- terized by V. Now looking into more detailed view, suppose that (s)( as differentiable loss function with prototype Ec(s) and (L) as its gradient during the optimization process, one step prototype rectification can be defined as an iterative process of Gradient Descent algorithm as described in equation 4. Symbol a denotes the learning rate, while w denotes (2-norm regularizer.
cess of prototype & can be defined as: (M) Recti
(c 1) = (s) - (V)( + (()
The previous study (Chen et al., 2018) discovered that the iterative process above can be viewed as Fuler discretization of an Ordinary Differential Equation (ODE). Therefore, the term VI((E())) in the process above can be derived into equation 5, where represents continuous variable such as time, and dār(a) ds represents continuous gradient flow of prototype #s(s) over s. Since the opti- mization process is executed by a neural network model gy (-) (last part of equation 5), then ODE
becomes Neural ODE,
L()(((>)) = dā.(s) = gw. S,,,
(S)
ds
We follow the implementation of GradNet (Zhang et al., 2022) as the neural network model (.) that executes the rectification process. Following the implementation on GradNet, the optimum pro- totype produced in the last step of rectification i.e. (M) = {0) + -.M Iv. (O), S, Q. s)). We follow the implementation of the previous study (Chen et al., 2018; Zhang et al., 2022), where the last term is solved by ODESolver based on Runge Kutta method (Alexander, 1990). Therefore, the optimum prototype (M) can be obtained by executing equation 6.
(6) In previous studies (Chen al., 2018; Zhang al., 2022), Q contains images from base task. Now dealing with limitations in FFSCIL, we can't afford to save exemplars from previous tasks or gather images from other clients. However, we have a shareable unified prototype Z, = 2, U 2, containing all learned class prototypes. Therefore, the uniqueness of our rectification is that generate
(M) = ODESolver (go,, Ea()), S, Q, s = M)
5

---

Published as a conference paper at ICLR 2025
support set S is constructed by drawing prototypes from (Hute ote2) where (Hic: oie 2) is computed by eq. 2-3 for all class c available in Ti', and generate query set Q by drawing prototypes from Mii, of) where (ft, of) is the property of & E Z,. As the implications, our rectification occurs fully in the embedding space and pseudo-rehearsal free. Note that Zi assigned by aggregated prototypes i.e. Zi Za in each federated round.
4.4 ADAPTIVE DUAL-HEAD CLASSIFIER FOR A BETTER PREDICTION
We design an adaptive dual head classifier by leveraging MIP classifier go(.) that works effectively in the highly labeled task (T") and sesoottpe-basedd (PB) classifier 9z (.) that has been proven robust in few shot tasks (T We combine both MLP and PB classifiers as one united head layer. MLP parameter & is optimized in the base task, while unified prototype set Z is optimized in all few shot tasks. In the testing phase, we deploy a head selector select which classifier to use for an input x. The head selector works by predicting the task ID based on the input-key matching. Given an input image x,, and sequence of prompt-key K K1 ...., K", we can find the task-id where x belong by finding the highest similarity of x and Kt L E [T]. The predicted label ý of an input is computed by equation 7.
[ollxxrrr (2)), ift=0 gz(feet pi)(t); otherwise,
where, 8 arg min C. K')
where L. is matching loss between input x and prompt-key K*. Note that equation is applicable both for the client and server side. The united classifier is expected to elevate the prediction result, rather than employing one of the FC or PB classifiers alone, this hypothesis will be proven in our ablation study.
4.5 FEDERATED LEARNING AND SERVER-SIDE AGGREGATION
Figure 2 shows that on a task-f, each client-/ optimizes its local parameters based on its local training data T Then the optimized parameters i.e. ((K) Pf), Dt, Zi) to the central server G are aggregated. We propose a simple weighted aggregation applying the principle "the more you learn, the better your knowledge". We consider the total participation of client S the task as the client's 's weight wn. In round r of task I, given clients that carry locally opti- mized parameter {(KK, Pf), 11 7))= and their weights {wi) then the aggregated parameter ((K', Pt), Di, ZG) is computed using equation 8.
(KK2, Pá), Pa, Ze) = Σt. Σ ((Kt, Di, Zi).us
(8)
In the base task (t = 0), the server aggregates and distributes prompt parameter (K), Pt) MLP head parameter &c. and unified prototype set Z& to all participating clients. while in the few shot tasks (t > 0), the server aggregates and distributes prompt parameters and unified prototype set only. The GradNet parameter V, is unshared and utilized only for local prototype rectification. Rather, client sends the rectified dynamic prototype 21 C Zi to the central server.
4.6 WRAP UP AND FINAL OBJECTIVE
In the base task, each client-l unifies fikp.pp) (x) with Z₁ then updates prompt and MLP head parameters i.e. (KD, PP) and $ In the few-shot tasks, client-l performs f(Kt,r9) (x) to form static prorotype Zies then construct support sample (S) and query samples (Q), and perform rectification for all âte E Zic. Afterward, it updates prompt parameters (Ki, Pf) and GradNet parameter V/, Therefore, the objectives of local training are defined in the equations below:
(i). Base Task (1 = 0) objective:
Li L.. (ga, ep, Zi), vi UC)) AL (Z, KP), (4., (.).
(9)
(ii). Few Shot Tasks (t > 1) objective:
8), Li>u = Lpee (gz. (9v. +,0. ACm Kl), (xi, 36) TT'
(10)
where fee denotes cross-entropy loss, fpee denotes prototype cross-entropy loss utilizing cosine distance as similarity measurement, L, denotes matching loss, and C is the label set of the unified prototype set Zı, (f(K).P;) denotes prompting output in eq. 1, and go, denotes rectification function in eq. 5. The detailed process of our method is presented in Appendix A.

---

Published as a conference paper at ICLR 2025
5 THEORETICAL ANALYSIS
Let θ = (P, V) be trainable parameters, F(O) = B[C(T: = E]C(T: V)) is the expected loss function, k, E, R. and Ls is local iteration, local epoch, global round, and number of selected local clients respectively. We follow L-smooth and μ-strongly convex F, G-bounded uniformly gradient assumptions, random uniformly distributed batches, and decreasing learning rate as in (Li et al., 201 (Bottou et al., 2018) We stated Theorem 1,2, and 3 as presented in Appendix B. Theorem and prove UOPP local training and federated convergence respectively, while theorem 3 proves UOPP generalization.
6 EXPERIMENTAL RESULTS AND ANALYSIS
6.1 EXPERIMENTAL SETTING
Datasets: our experiment done using three henchmarks i.e. split CIFARIOO, split MinilmageNet, and split CUB200. The CIFAR 100 and minilmageNet datasets contain 100 classes while CUB200 is a dataset of 200 classes. We follow the settings from (Tao et al., 2020a) for tasks and classes per task split, and (Dong et al., 2023) for the federated setting. For CIFAR 100 and MinilmageNet, we split the dataset into 9 tasks i.e. 60 classes for the base task (t = 0), and classes for each few-shot task (t 2 1). We split the CUB200 dataset into 11 tasks i.e. 100 classes for the base task, and 10 classes for each few-shot task. Few shot tasks are measured in 5-shot and 1-shot settings.
Benchmark Algorithms: UOPP are compared with 9 benchmark algorithms i.e. LGA (Dong al., 2023), TARGET (Zhang et al., 2023b), LANDER (Tran et al., 2024), and Fed-CPrompt (Bagwe et al., 2023) that represent the SOTA in federated class incremental learning, Fed-S3C(Kalla & Biswas, 2022) that represents SOTA in few shot class incremental learning, Fed-L2P(Wang et al., 2022b), Sod-DualP(Wang et al., 2022a), Fed-CODAP(Smith et al., 2023), and PILoRA(Guo al., 2024) that represent SOTA in FCIL. Except for Fed- CPrompt "Fed-" " denotes that the method is cus- tomized in a federated manner from its original (stand-alone) mode, by using FedAvg as the aggre- gation function. We only run LANDER and PIL.RA for CIFAR 100 since their official code, setting, and pre-trained embedding that can be executed in our setting is only for CIFAR 100 dataset. We also evaluated our method in standalone FSCIL and compared it to FSCII SOTAs i.e. THEN et al., 2024), NC-FSCIL Yang et al., 2023), OrCo(Ahmed et al., 2024), and Pri PriViLegePPark et al., 2024), Please see Appendix D.I.
Details and Metrics: our numerical study is executed under a single NVIDIA A GPU with 40 GB memory across 3 different random seeds. Adapted from (Dong al., 2023), the simulation is run by 20 total clients and global server, where in each round, 6 (30%) local clients are selected randomly. Each client randomly receives 60% (1) = 0.6) classes. The total global round is set to 90 (10 rounds/task) for CIFAR100 and MinilmageNet and 110 for CUB200. Our task split setting is different from the recent study (Jiang et al., 2024), since it follows FCIL setting, while our setting follows FSCIL setting. We evaluate the consolidated algorithms for all learned classes with accuracy metries (Acc.) adn performance drop (PD). Besides, we measure the accuracy of base classes, novel classes, and harmonic mean accuracy that represents stability stability-plasticity performance. Please see Appendix F for the detailed experiment settings, hyperparameters, and metrics.
6.2 MAIN RESULTS
a) General Performance:
the numerical result of the consolidated algorithms is shown in table 1. The proposed method (UOPP) achieves the highest accuracy with a significant gap 5 76% compared to the competitor methods both in 5-shot and 1-shot settings. Fed-S3C, TARGET, LGA, and LANDER achieve relatively low performance with 30 76% gap in benchmark datasets in both 5-shot and I-shot settings compared to UOPP. The results confirm that the FCIL and FSCIL methods can't 't be applied directly on FFSCIL. Meanwhile, prompt- based methods i.e. Fed-I.2P, Fed- DualP, Fed-CODAP, and Fed-CPrompt achieve a relatively better performance than those 4 methods. Compared to UOPP, the methods have relatively smaller gap i.e. 5 - 36%. The results prove that prompt-based methods are more promising than the SOTAs of FCIL and FSCIL, The proposed method also achieves the lowest performance drop with (0.7 - 19%) followed by Fed-L2P with (0.4-22% PD and Fed-Duall with (-0.4- 28% PD. Despite utilizing ViT and prototype approach, PILoRA achieves lower performance than the prompt- proset-based based method i.e. < 52% in average. Looking per-task performance, Figure 4 shows that UOPP achieves the highest accuracy in all tasks in those
7

---

Published as a conference paper at ICLR 2025
Table 1: Numerical result of the consolidated algorithms in CIFART00, MinilmageNet, and CUB200 dataset in 5-shot and I-shot setting across 3 different seeded runs.
(a) Complete numerical result on CIFAR 100 datasct with 5-shot setting
Method
Trainable
Accuracy in each session (%)
Avg!
PD +
Gap!
Params.
Fed-S3C
CNN
44.51
48.97
47.77
45.35
43.48
41.47
40.33
39.32
37.71
43.21
6.80
46.80
TARGET
CNN
68.90
63.61
59.06
55.12
51.68
48.64
45.94
43.52
41.34
53.09
27.56
36.92
LGA
CNN
73.76
69.80
65.59
60.26
56.87
52.94
50.66
47.69
44.89
58.05
28.87
31.96
LANDER
CNN
58.60
61.75
56.26
52.11
47.71
44.71
41.69
40.28
38.87
49.11
19.73
40.90
Fed- L2P
Prompt
73.47
74.20
73.37
71.88
70.72
68.66
68.37
71.20
18.81
Fed-DualP
Prompt
76.39
82.75
83,37
80,80
79,93
78.26
77.73
76.98
77.11
79.26
-0.72
10.75
Fed-CODAP
Prompt
81.73
69.29
70.81
68.67
67.17
66.14
64.32
64.79
64.12
68.56
17.62
21.45
Fed CPrompi
Pronpt
88.00
64.63
69.30
67.39
63.39
62.33
61.11
59.78
$9.00
66.10
29.00
23.91
ORA
Params (A.B)
67.40
62.22
57.77
53.92
50.55
47. 58
44.93
42.57
40.44
51.93
26.96
38.08
UOPP (Ours)
Prompt
90.57
90.58
90.85
90.96
91.23
91.51
91.56
91.74
81.05
90.01
9.52
0.00
(b) Summarized numerical result on CIFAR 10D dataset with 1-shot setting, Minilmagenet, and CUB200
dataset with 5-shot and 1-shot settings
CIFARIO0
MinilmageNet
CUB200
Method
L-shol
5-shol
5-shol
Avg
PD)
Gap
Avg
P1)
Gap
Avg
PD)
Gap
Avg
PD
Gap
Avg
PD)
Gap
F3C
41.97
9.07
46.65
29.73
5.48
63.19
29.16
8.24
63.05
14.91
7.22
65,89
14.40
7.85
62.33
TARGET
53.09
27.56
35.53
44.77
23.24
48.15
44.77
23.24
47.44
21.47
16.77
59.33
20.70
14.17
56.03
LGA
58.29
24.80
30.33
35.07
30.51
57.85
31.65
28.44
60.56
16.92
14.16
63.88
10:02
17.86
66.71
LANDER
48.43
19:90
Fed-T.2P
74.29
3.75
14.33
78.92
1.01
14.00
80 80
0.45
11.41
58.26
22.67
22.54
57.12
24.34
19.61
Fed-DualP
81.95
-0.87
6.67
85.91
-0.45
7,01
86.88
0.67
5.33
62.89
28.21
17.91
62.41
26.94
14.32
Fed-CODAP
68.72
21.62
19.90
80.11
15.12
12.81
80.13
15.70
12.08
37.55
42.26
43.25
39,80
45.68
36,93
Fed.Cr CPrrmpt
73.82
24.69
14.80
88.77
8.29
4.15
86.34
12.38
5.87
61.23
37.33
19.57
58.26
36.86
18.47
PII oRA
51.78
26. 88
36.84
UOPP (Ours)
88.62
5.83
0.00
92.92
0.73
0.00
92.21
2,18
0.00)
80.80
10.90
0.00
76.73
19.54
0.00
CIFAR100 S-shot
MinllmageNeet 5-shot
CUB200 S-shot
CIFAR100 1-shot
MinilmageNee 1-shot
CU8200 1-shot
80
75
60-
60
60
40
5
Sassion
Saesiion
Swesion
Sension
Fed-S3C
TARGET
USA
LANDER
Fed
Fied-DusiP
FediCCODAP
FediCPrompt
PILORA
UOPP
Figure 4: Visualization of the performance of consolidated algorithms in MinilmageNNet CIFARI00
and CUB200.
three datasets. In the first task (base task), the proposed method achieves higher performance with a small gap i.e. 1 - 2% than the competitor methods. However, with the increasing number of tasks, the gap gets higher e.g. 2 3% in task- 2 4% in task-2, and 2 6% in task-9 (last task). shows that the proposed method handles catastrophic forgetting better than the competitor methods. Please see our extended analysis on different k-shot and standalone FSCIL in Appendix D. The complete numerical results for all dataset are presented in Appendix H.
b) Stability-Plastiiity Analysis: We evaluate UOPP stability-plastiiity perfor- mance by evaluating the harmonic mean accuracy each few-shot task. Ta- ble 2 shows the harmonic mean accuracy of consolidated methods in CIFAR 100 with 5-shot setting, while Figure 5 vi- sualizes the accuracy for base classes. novel classes, and harmonic mean accu- racy. Both show that UOPP achieves the best harmonic mean accuracy with 2 15% gap on each task and 18-58% 58% on average of all tasks. The results prove that UOPP handles stability- plasticity dilemmas better than its competitors. Looking at the base classes and novel classes perfor- mance, Figure 5 shows that UOPP achieves the highest accuracy of base classes and novel classes that are consistent through all tasks. Fed-L2P has increased base classes and novel classes accuracy, but the performance is still below UOPP with a significant gap. Fed-Dual, Fed-CODA CODAP, and Fed- CPrompt have relatively stable base class accuracy but decreasing novel class accuracy. TARGET,
Table 2: Harmonic mean accuracy of the consolidated algo-
rithms in CIFAR100 dataset with 5-shot setting.
Method
Harmonic Mean Acc. by Session (%)
Avg
PD
Gap
2
Fed-s3C
49.3
42.8
38.0
35.4
3399
33.4
32,9
32.1
37.2
17.3
53.3
TARGET
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
90.5
44.7
31.5
12.3
99
71.2
LANDER
0.0 0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
90.5
Fed-1.2P
46.0
50.2
49.2
54.8
56.1
57.7
.1.7
38.3
Fed-DualP
67.4
71.4 66.8
68.4
68.2
69.3
70.1
71.8
69.2
-45
21.3
Fed-CODAP
69.8
69.3 66.1 65
63.3
61.8
62.0
61.3
64.9
85
25.6
Fed CPrompt
73.3 74.4 67.6
63.5
62.3
60.4
59.4
58.9
65.0
14.3
25.5
PILORA
0.0 0.0 0.0 0.0 0.0 0.0
0.0
0.0
0.0
0.0
90.5
UOPP 90.6 91.6
92.0
0.0
8

---

Published as a conference paper at ICLR 2025
Classes Accuracy (5-shot)
Novel Classes (5 shot)
Harmanic Accuracy (5ssho))
Base Classes Accuracy (1:shot)
Novel Classes Accurasy (1-shot)
Harmonic
(1-shot)
lecunseyeng
80
50
40-
60
60
20
3.0
Fed-S3C
TARGET
LGA
LANDER
Ped-L2P - Fed--uuil
Fed- COD-C
PediCPrompt
PILORA
UOPP
Figure 5: Visualization of the performance of consolidated methods for base classes and novel classes in
CIFAR 100 Dataset.
Table 3: Summary of the numerical result of the consolidated algorithms in CIFAR 100 dataset with the varia-
tion of Non-i.i.d level (a), selected local clients (b) and variation of total global round (c).
(a) Non-Lid level
(b) Selected Local Client (L)
(e)
Total Global Round (R)
Method
1=0.66
n=0.4
1=0.2
L=4
L=6
L=8
R=$4
R=72
R=90
(40%)
(20%)
(30%)
(40%)
r/task)
r/task)
(10 ritask)
Avg
PD
Avg
PD
Avg
PD
Avg
PD
Avg
PD
Avg
PD
Avg
PD
Avg
PD
Avg
PD
Fed-S3C
43.2
6.8
34.2
0.5
18.9
1.4
43.9
4.2
43.2
6.8
42.7
6.2
43.4
5.3
47.4
10.1
42.6
6.3
TARGET
27.6
41.3
27.5
51.4
27.6
56.7
44.4
26,9
LGA
58.1
28.9
56.4
29.5
56.4
29.7
56.3
30.4
58.1
28.9
57.9
29.1
57.1
22.1
55.2
26.3
58.0
29.1
LANDER
49.1
19.7
18.9
47.4
22.3
49.1
19.7
51.6
20.8
37.9
31.4
47.2
26.0
49.1
Fed-DualP
79.3
-0.7
60.2
-1..
6.6
-21.5
76,2
-11.2
79.3
-0.7
79.2
11.1
81.5
-0.9
80.2
1.8
79.3
-0.7
Fed-Cpompt
66.1
29.0
66.2
28.1
66.2
28.1
62.5
36,9
66.1
29.0
59.6
38.1
62.5
36,9
59.6
38.1
66.1
29.0
PILaRA
51.9
27.0
43.4
22.5
35,3
18.3
50.6
26.3
51.9
27.0
51.1
26.5
55.9
29.0
53,9
28.0
51.9
27.0
UOPP
90.0
9.5
86.3
4.3
68.8
29.4
89.1
4.2
90.0
9.5
91.3
0.0
90.7
-1.
90.8
0.2
90.0
9.5
LANDER, and PILoRA can't achieve plasticity since their novel classes' accuracy is close to O. Fed-S3C maintains stability- dilemma with relative balances, but the performance of both components is low (< 40%). The complete numerical results are presented in Appendix
6.3 ROBUSTNESS, ABLATION, AND FURTHER ANALYSIS:
a) Different Non-i.i.. level: Table (a) shows the performance of the methods w.r.t. non-i.i.d level represented by the percentage of available class (lower is harder). The table shows that our method outperforms the existing methods in all non-i.i.d. levels with significant margin iee up to 50%. The table shows that the small available class (1) = 20%) remains challenging since all the methods achieve less than 70% on average. In contrast, in other cases i.e. (1) > 40%) our method achieves > 86% accuracy average.
Table 4: Summary of the numerical result of our abla-
tion study CIFARIOO dataset, MLP, and PB and Rect. denote prototypes-bet
and rectification respectively.
Stiatic Dynumic MLP PB. Rect.
Conf.
Proto. Proto.
Head Head
Asg
PD
Gap Time
A
v 80,62
7.76 9.39
6.01h
C
v v 88.72
5.49
1.29 8.00h
69.23
37.76
20.78 3.50h
UOPP
V v 90.01
9.52 0.00
6.05h
b) Different Participating Local Clients: we evaluate UOPP robustness in various selected local clients simulating fluctuations of participating clients in real-life applications. Table 3 (b) shows the performance of consolidated methods with 4 (20%), 6 (30%), and 8 (40%) selected local clients from 20 total clients. The table shows that the UOPP achieves the highest performance in all combinations with a significant gap i.e. 2 10% compared to the competitor methods. Table also shows that UOPP achieves higher performance on a higher percentage of participating local clients. Besides, UOPP achieves a relatively lower performance drop (PD) than Fed-S3C, TARGET, LGA, LANDER and Fed-CPrompt. In 8 (40%) local client cases, UOPP achieves the lowest PD, while in (20%) and 6 (30%) local client cases, UOPP experiences a higher PD than 8 and 4 local client cases due to an accuracy drop on the last task. The accuracy drop is caused by a mismatch between the samples and prompt keys resulting in inaccurate feature extraction and classifiers selection. The more detailed per-task result presented in Appendix
c) Variation of Total Global Rounds: we evaluate UOPP robustness in smaller global rounds sim- ulating real-world conditions where the global model is urgently needed, thereby requiring smaller rounds. Table 3(b) summarizes our investigation on 54 (6 r/task) to 90 (10 r/task) global rounds in CIFAR100 datasets, while the complete result is presented in Appendix J. The table shows that UOPP achieves the highest performance with significant gap i.e. (2 9%) compared to the competi- tors. In the lower global rounds, the UOPP achieves even better performance than in normal global rounds it doesn't experience the accuracy drop aforementioned.. UOPP also achieves the smallest.

---

Published as a conference paper at ICLR 2025
PD compared to the competitor methods in smaller rounds. Prompt-based method i... Fed-DualP is proven to be more robust than Fed-S3C, TARGET, LGA, LANDER, and Fed-CPrompt. Further- more, those 3 methods achieve lower accuracy in the smaller global rounds. This finding confirms the robustness of our proposed method in the case of low global rounds.
we conduct an ablation study investigate the contribution of each component of the proposed method. The result is summarized in Table 4, while the detailed result is presented in Appendix K. The result shows that the absence of static prototype (Conf. A) and dynamic pro- totype (Conf. B) drops the average performance with 9.4% and 4.7% gap respectively. This result proves the importance of unified prototypes for prompt learning to deal with FFSCIL problems. The absence of the MLP head drops the performance with 1.3% gap. This result shows the presence of MLP classifiers (Conf. C) contributes to the model prediction performance. Last, the absence of the prototype-based head (Conf. D) drops the performance with the most significant magnitude e.g- 20.8%. It proves that the PB classifier of rectification (Rect.) follows the presence of dynamic prototype (e.g. in configuration B and D where dynamic prototype is absent, the rectification is absent in those configurations).
d) Ablation Study:
must to deal with few-shot tasks. Note that the presence
e) The Importance of Prototype Rectification: We analyze the impact of prototype rectification in our proposed method. Figure 6 shows the validation loss of novel classes with rectification and without rectifi- cation on CIFAR 100 dataset with 5-shot and 1-shot settings. The figure shows that, without prototype rectification (red line), our method produces a far higher validation loss for novel classes as there are many misclassifidd samples. On the contrary, with prototype rectification (blue line), our method produces far smaller and more stable validation loss. The figure also shows that the difference in loss magnitude between the two variants is even higher in the --shot setting. This finding emphasizes the importance of prototype rectification in our method.
Novel Classes Loss on CIFAR1 S-shot
Novel Classes Loss on CIFAR160 1-shot
2.5
Without Rectifiction With Rectflication
1.0
0.5
Session
Figure 6: Visualization of Novel Classes Valida- tion Loss with vs without Rectification.
Complexity, Running Time, Parameters, and Communication Cost: our complexity analysis shows that our proposed method has the same complexity i.e. O.R.L.N) where R is the number of global rounds, L is the number of participating local clients in each round, and N is the size of the training data in each client. The detailed complexity analysis is provided in Appendix C. Table 5 shows the training time of the consolidated method in CIFARI00 dataset. The table shows that the proposed method re- quires a moderate training time since is lower than LGA and higher than the other methods. In the base task, UOPP trains only small amount (0.33 M) of parameters, since it trains only its
Table 5: Comparison of Parameters, Communication Cost, and Training time CIFARION 100 dataset.
Number Parameters (M)
Comm.
Running Time (h)
Method
Trainable
Sharable
Cost (MB)
Base
F$
Base
FS
Base
FS
Task
Task
Task
Task
Task
Task
Fed TARGET
11.7
11.7
11.7
11.7
46.8
46.8
4.1
11.3
11.3
17.45
17.45
61.7
61.7
2.02
LGA
11.3
11.3
15.42
15.42
69.8
69.8
8.22
LANDER
11.3
11.3
17.45
17.45
61.7
61.7
4,78
Fod-L2P
0.29
0.29
0.29
0.29
1.16
4.83
Fed-DualP
0.33
0,33
0.33
0.33
1.32
1.32
3,2
Fed CODAP
0.33
0.33
Fed Ped-Cprompe
0.33
0.33
0.33
0.33
1.32
1.32
2.03
PILORA
0.30
0.30
0.34
0.37
1.37
1.50
6.24
UOPP
0.33
34,1
0.38
0.41
1.5
1.63
6.05
prompts and MI. classifier. However, in a few-shot tasks, it trains GradNet for prototype rectifica- tion which contributes to the high amount of trainable parameters. However, UOPP keeps its low communication cost both in the base task and few shot tasks since it only shares prompt+MLP or prompt+ prototypes in both tasks. Please see Appendix D and E for our extended analysis on running time, memory consumption, limitations and potential solution.
CONCLUDING REMARKS
We define a new Federated Few-Shot Class-Incrementis Learning (FFSCIL) problem and develop a novel Unified Optimized Prototype Prompt (UOPP) model that utilizes task-wise prompt learning to mitigate task interference empowered by shared static-dynamic prototypes, adaptive dual heads, and weighted aggregation. The dynamic prototype tackles prototype bias by iterative rectifications. Our comprehensive experimental results show that UOPP significantly outperforms existing SOTA methods of FCIL, FSCIL, and CIL, on three datasets with a significant gap i.e. up to 76%. Our deeper analysis confirms that the proposed method achieves better mability-plasticin trade-off, and robustness in different local clients and small global rounds. Our analysis shows that our proposed method requires moderate training time but lower communication cost than the SOTAS.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
M. Anwar Ma' sum acknowledges the support of ookoopedia-UU Centre of Excellence for GPU access run the experiments.
REFERENCES
Noor Ahmed, Anna Kukleva, and Bernt Schiele. Orco: onality and contrast for few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 28762-28771, 2024.
Towards better generalization via orthog-
Roger Alexander. Solving ordinary differential equations i: Nonstiff problems (e. hairer, sp norsett, and g. wanner). Siam Review, 32(3):485, 1990.
Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) forget. In Proceedings of the European confer- ence on computer vision (ECCV), pp. 139-154, 2018.
Gaurav Bagwe, Xiaoyong Yuan, Miao Pan, and Lan Zhang. Fed-eprompt: Contrastive prompt for rehearsal-free federated continual learning. arXiv preprint arXiv:2307.04850 2023.
Léon Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM review, 60(2):223-3 311, 2018.
Pietro Buzzega, Mattco Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark ex- perience for general continual learning: strong, simple baseline. Advances in neural information processing systems, 33:1 15920-15993,, 2020.
Francisco M Castro, Manuel J Marín-Jinnn Marín-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteck Alahari. End-to-end incremental learning. In Proceedings of the European conference computer vision (ECCV), pp. 233-248, 2018.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Jiahua Dong, Lixu Wang, Zhen Fang, Gan Sun, Shichao Xu, Xiao Wang, and Qi Zhu. Federated inanemiacremetall learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10164-10173 10173, 2022.
Jiahua Dong, Hongliu Li, Yang Cong, Gan Sun, Yulun Zhang, and Lue Van Gool. No one left behind: Real-world federated class-incremental learning. JEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-17, 2023. doi: 10.1 10.100//TAAM.22033333222
Weifeng Ge.
Deep metric learning with hierarchical triplet loss. In Proceedings of the European conference on computer vision (ECCV), pp. 269-285, 2018.
Dipam Goswami, Bartlomiej Twardowski, and Joost Van De Weijer. Calibrating higher-order statis- tics for few-shot -las-inncrementll learning with pre- pre-trained vision transformers. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 4075-4084, 2024.
Haiyang Guo, Fei Zhu, Wenzhuo Liu, Xu- Yao Zhang, and Cheng-Lin Liu. Pilora: Prototype guided incremental lora for federated dass-increment learning. In Proceedings of the European Con- ference on Computer Vision, 2024.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF conference Computer Vision and Pattern Recognition, pp. 831-839, 2019.
Yalan Jiang, Yang Cheng, Dan Wang, and Bin Song. Rra-ffscil: Inter-intra classes representation and relationship augmentation federated few-shot incremental learning. Neurocomputing, pp- 127956, 2024.
Jayateja Kalla and Soma Biswas. S3c: sulf-super stochastic classifiers for few-shot class- incremental learning. In European Conference Computer Vision, pp. 432-448. Springer, 2022.
11

---

Published as a conference paper at ICLR 2025
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska Barwinska, et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-35) 2017.
Muhammad Rifki Kurniawan, Xiang Song, Zhiheng Ma, Yuhang He, Yihong Gong, Yang Qi, and Xing Wei. Evolving parameterized prompt memory for continual learning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 13301-13309, 2024.
Steinar Laenen and Luca Bertinetto. On episodes, prototypical networks, and few-shot learning. Advances in Neural Information Processing Systems, 34:24581-24592 2021.
Gwen Legate, Nicolas Bernier, Lucas Page-Caccia, Edouard Oyallon, and Eugene Belilovsky. Guid- ing the last layer in federated learning with pre-trained models. Advances in Neural Information Processing Systems, 36, 2024.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and sys- lems, 2:429-450, 2020.
Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data. arXiv preprini 18Xiv:190.002/89, 2019.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint 1rXxi:2011.001900 2021.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis
and machine intelligence, 40(12):2935-2947 2017.
Muhammad Anwar Ma' sum, Mahardhika Pratama, Savitha Ramasamy, Lin Liu, Habibullah Habibullah, and Ryszard Kowalczyk. Pip: cenootyes--ijjeteed prompt for federated class in- cremental learning. CIKM '24 pp. 1670-1679. Association for Computing Machinery, 2024.
ISBN 9798400704369. doi: 10.1 10.1145/36226733367999
Pratik Mazumder, Pravendra Singh, and Piyush Rai. Few-shot lifelong learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 2337-2345, 2021.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficeeett learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273-1282. PMLR, 2017.
Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with aux- iliary classifier gans. In International conference on machine learning, pp. 2642-2651. PMLR, 2017.
Keon-Hee Park, Kyungwoo Song, and Gyeong- Park. Pre-trained vision and language trans- formers are few-shot incremental learners. In Proceedings of the IEEE/CV Conference on Com- puter Vision and Pattern Recognition, pp. 23881 -23990, 2024.
Ameya Prabhu, Philip HS Torr, and Puneet K Dokania. Gdumb: A simple approach that questions our progress in continual learning. In Computer Vision-ECCC 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II 16, pp. 524-540. Springer, 2020.
Daiqing Qi. Handong Zhao, and Sheng Li. Better generative replay for continual federated learning. arXiv preprint arXiv:2302.13001 2023.
Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.
Anurag Roy, Riddhiman Moulick, Vinay K Verma, Saptarshi Ghosh, and Abir Das. Convolutional prompting meets language models for continual learning. In Proceedings of the IEEE/CVF Con- ference Computer Vision and Pattern Recognition, pp. 23616-23626 2024.
12

---

Published as a conference paper at ICLR 2025
Guangyuan Shi, Jiaxin Chen, Wenlong Zhang, Li-Ming Zhan, and Xiao-Ming Wu. Overcoming catastrophic forgetting in incremental few-shot learning by finding flat minima. Advances in neural information processing systems, 34:6747-6761, 2021.
Hanul Shin, Jung Kwon Lee, Jachong Kim, and Jiwon Kim. Continual learning with deep generative replay, Advances in neural information processing systems, 30, 2017.
James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Boninn Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and 7solt Kira. Coda-prompt: Continual de- composed attention- based prompting for rehearsal -free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11909-11919, 2023.
Yue Tan, Guodong Long, Jie Ma, Lu Liu, Tianyi Zhou, and Jing Jiang. Federated learning from pre-trained models: A contrastive learning approach. Advances in neural information processing systems, 35:19332-19344, 2022.
Xiaoyu Tao, Xiaopeng Hong. Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong. Few- shot class-incremental learning. 2020 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12180-12189, 2020a.
Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong. Few- shot class incremental learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 12183-12192, 2020b.
Minh-Tuan Tran, Trung Le, Muan-May Le. Mehrtash Harandi, and Dinh Phung. Text-enhanced data-free approach for federated class-incrent claseinncremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 28870-23880, 2024.
Anastasiaa Usmanova, François Portet, Philippe Lalanda, and German Vega. A distillation-basee ap- proach integrating continual learning and federated learning for pervasive services. arXiv preprint 10904:2/09.04197,, 2021.
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al. Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016
Qi- Wang, Da-Wa Zhou, Yi-Kai Zhang, De-Chuan Zhan, and Han-Jia Ye. Few-shot class- incremental learning via training-free prototype calibration. Advances in Neural Information Processing Systems, 36, 2024.
Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European Conference on Computer Vision, pp. 631-648. Springer, 2022a.
Zifeng Wang, Zizhao Zhang, Chen- Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vin- cent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Pro- ceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 139-149, 2022b.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 374-382, 2019.
Yibo Yang, Haobo Yuan, Xiangtai Li, Zhouchen Lin, Philip Torr, and Dacheng Tao. Neural collapse inspired feature-classifie alignment for few-shot fass-incremennal learning. In The Eleventh International Conference Learning Representations, 2023. URL https:ttt///ec. /taps////////eeee./// net/ fort/foorm?2 iididddtto
Jachong Yoon, Wonyong Jeong, Giwoong Lee, Eunho Yang, and Sung Ju Hwang. Federated contin- ual learning with weighted inter-elient transfer. In International Conference on Machine Lear- ing, pp. 12073-12086 PMLR, 2021.
13

---

Published as a conference paper at ICLR 2025
Baoquan Zhang, Xutao Li, Shanshan Feng, Yunming Ye, and Rui Ye. Metanode: Prototype op- timization as a neural ode for few-shot learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 9014-9021, 2022.
Chi Zhang, Nan Song, Guosheng Lin, Yun Zheng, Pan Pan, and Yinghui Xu. Few-shot incremental learning with continually evolved classífiers. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 12455-12466 2021.
Jie Zhang. Chen Chen, Weiming Zhuang, and Lingjuan Lv. Addressing catastrophic forgetting in federated class-continual learning. arXiv preprint (rxxix23333069333 2023a.
Jie Zhang, Chen Chen, Weiming Zhuang, and Lingjuan Lyu. Target: Federated class-continull learning via exemplar-free distillation. In Proceedings the IEEE/CVF International Conference Computer Vision, pp. 478.478244777 2023b.
Junting Zhang, Jie Zhang, Shalini Ghosh, Dawei Li, Serafettin Tasci, Larry Heck, Heming Zhang, and C-C Jay Kuo. Class-incremental learning via deep model consolidation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131-1140, 2020.
Hanbin Zhao, Yongjian Fu, Mintong Kang. Qi Tian, Fei Wu, and Xi Li. Mgsvf: Multi-grained slow versus fast framework for few-shot class-incremental learning. IEEE Transactions Pattern Analysis and Machine Intelligence, 4663):157-15588, 2024. doi: 10.1109/PAAM.22213333333
A DETAILED PROCESS UNIFIED OPTIMIZED PROTOTYPE PROMPT (UOPP)
In this section, we present the detailed algorithm of UOPP as shown in algorithm 1.
B DETAILED THEORETICAL ANALYSIS
V)) is the expected loss function, k, E, R. and Ls is local iteration, local epoch, global round, and number of selected local clients respectively. We follow L-smooth and j-strooglyy convex F, G-bounded uniformly gradient assumptions, random uniformly distributed batches, and decreasing learning rate as in (Li et al., (Bottooo et al.. 2018) We state the following theorems.
Let θ == (P, क; V) be trainable parameters, F(0) [) = EiC(T;
Theorem 1: lim inf o0 E V ( [+) =0
Theorem 2: Let @1 O O* is the initial, last updated (/8-th,, and optimum parameter respectively,
F' is minimum of F, exist A, B, C, > 0 so that: = < 4 (1 - -0
Theorem 3: Given O* and ④ are optimal parameter in T, UZ and T respectively, where T C CT, where IT, = E (0, 1), then at least there's c >> 0 that satisfy F(O,T") F(O":T) c. Theorem 1 and prove UOPP local training and federated convergence respectively, while theo- rem 3 proves UOPP generalization. The detailed theoretical analysis, assumptions, and proofs are presented in Appendix B.
Let θ (P, Φ, w) be the trainable parameters, F(O) = E[C(T θ) = B[C(T: L v) is the expected loss function, k, E, R, and is local iteration, local epoch, global round, and number of selected local clients respectively. Please note that in this analysis. L denotes the number of selected local clients, while 2 denotes a constant for the /-smooth coefficient. Following the update rule in section 4.3, the expression of F(O) above can be detailed as follows:
(i) Base Task (t = 0): = 2). and F()) = E[C(T: θ)] = E|C, (T: (P, as local clients update (P, b)) using Li+ following equations 7 and 8.
(i) FS Task (t 2 1): θ = (P, v) and F(0)= = E[C(T: θ)] = E[Cr/+ (T; ()) as local clients
update (P, v using Lifs+ following equations 9 and 10.
We adopt the SGD optimization convergence analysis (Bottou et al., 2018) and FedAvg convergence analysis (Li al., 2019) assumptions follows:
14

---

Published as a conference paper at ICLR 2025
Algorithm UOPP
1: Input: Number of clients Luul, number of selected local clients L,, total number of rounds R, number of task 7 + 1, local epochs E, batch size B.
2: Distribute frozen VIT backbone f to all clients {4) and central server
Initiate prompt, key, and head layer for all clients and central server Pg Pl, & D/, Vi init(), {1.
4: Rr t- R + 1), Rt represents round per task
5: Init global and local unified prototypes Za Zí, 7 = 0
6: for 0 10:T'm do
7:
for ror-=1 Rr do
8:
{1)= f randomly select local clients from Lall total clients
9:
Clients execute:
10:
if R = then
11:
Compute static prototype 2i as in Eq. (3)-(4), then send it to server end
12:
Receive global parameters i.c. prompt, FC FC layer, and prototypes set Pa, PG. and Zg
14:
Assign local parameters (P,, 4; Zi) t (Pa, 2G, ZG)
B -Split T into sized batches
16:
for = 1 do
for b = do
18:
if (1 0) then // Base Task Update
Compute prompt-generated feature S(KT pf)(()))))))))))))) as in Eq. (2)
20:
Compute logits with FC clsasifier go, (f(()))))))))) (x) U Za)
21:
Compute loss Lı=u as in Eq. (10)
22:
Update local parameters (K Pt,, hb) based on L
23:
else (1 >> 0) // Few-shot Task Update
24:
Compute static prototype Zi using feature f(K3, pP) (x) as in Eq. (2)
25:
Draw S from Ý, and draw Q from Zi ZG
26:
Rectify dynamic prototype Zi using gv (-)as in Eq. (5) to (7)
27:
Form unified prototype Zi = Za Zi
28:
Compute logits with PB classifier 9z, (f()))))) US)
29:
Compute loss Le>o as in Eq. (11)
30:
Update local parameters (Kt Pt M1) based on Li>0
31:
end if end for
33:
if t = 0 then
34:
Update local static prototype 2, as Eq. (3)-(4) for class e Ti
35:
end if end for
36:
37:
if =
0 then
38:
Set unified prototype Z, Â. J 2i
39:
else
40:
Set unified prototype Z, Żg u 2,
41:
end if
42:
Store local parameters (K1, Pf; D1, V1, 21)
Compute clients' weight wi
44:
Send local parameters (K Pt Di, Z,) and weight wf to server
Server executes:
46:
if Rr = then
47:
Receive clients initial static prototype 21 for E [I..L]
48:
Generate ZG = Za U Ag for E [I..L) and send Zg to clients
49:
end if
50:
Receives selected clients parameters (K₁, Pt, t,, Zi) and weight wi for E [1.
51:
Do weighted aggregation in Eq. (9)
52:
Send global parameters (K&, Pt PG. Zg) to clients for the next round
53:
end for
54:
end for
Output: Optimal Global parameters (Po, Pc, Zc)
15

---

Published as a conference paper at ICLR 2025
Assumption 1: Fi....F,. FLs are all L-smooth: for all @ and O Fi(O) & Fi(O") + (0
()) + / 1
Assumption 2: Fi....F....... Fig are all - strongyyly convex: for all and O', () S Fi((") +
(0 -') VF(()) _ e
Assumption 3: Let e be the random uniformly sampled from 1-th local data at k th itera- tion.. The variance of stochastic gradients in each client is bounded by the following criteria: BI (Ff) of for I 1,2,..... Ls
Assumption 4:The expected squared norm of stochastic gradients in each client is bounded by: BI 4f]l S G² for all 1,2,.., Ls and 1,2,..... K where K E N.
Assumption 5: Σk=1 of oo and Ek=1 at < oc where of is the learning rate of ( - th client in k-th step training.
We follow the theoretical analysis in federated class incremental learning method (Ma 'um et al., 2024) as it has similar characteristic i.e. prompt-based method supported by shared prototypes.
B.1 PROOF OF THEOREM
Let client-l be trained locally with its local data T/ U Z. where T/' is localty observed training samples for 1-th task and 7 = 7 = Zg aggregated unified prototype for task t shared by server respectively. We assume that Z is augmented so that | ~ | for Ze, Z and x"_ E Ti T As an implication, the number of prototypes of unavailable classes in T, and the samples of available classes in 7," are balanced. Then the local model θ (Pr, b1) or θ = (Pi, Vz) is updated in K iterations based minibatches drawn from TZZZ. Since thefeature extractor parameters are frozen, and T7' Z has balance samples for all classes, then E approximates ę that sample from T The local model parameters are optimized by stochastic gradient (SG) approach. Suppose that g( g(O,, </) a SG function, then the parameter update can be simplified as:
o +1 + o at g ,
(A11)
Applying assumption 1, and local parameter updates θ by iterating stochastic gradient with sample & then we get:
) s - e/)" + 1100 e/1
(A12)
[ <t) a 11a(@)
The equation above can be derived into:
E -m(t)) - -etv
(A13)
+ + 22 T. b c @ &t )
The inequation above shows Of optimization by SG method at step k. and it shows the reduction of Fi (left side) is bounded by the value in the right side involving V which is derivative of Fi at Of along with -g(0) <f) (first term) and second moment of g g </) (second term). Let g(O et) be the unbiased estimator Fi, then we derive inequation above into:
Bey [I(OO) Fi(e)) - (
42
Bey (llg E))
(A14)
The inequation above guarantees SGD convergence as long as the stochastic directions and stepsize are chosen. We apply the restriction below to avoid the harm of the second term of the right side in the inequation above,
Vig(O/. <h) [lllo(ff < [R ef)]1
(A15)
16

---

Published as a conference paper at ICLR 2025
Adopting first and second-moment limit in (Bottou et al., 2018), then add the following sumption.
Assumption 6: The objective function Fi and SG satisfy the following conditions.
(a). The sequence of {Of) is contained in an open space where Fi is bounded below by a scalar Fin
(b) Exist scalars VG 2 v > 0 so that for all E satisfy:
Vn(()) (o(o) <h) 2 = and
(A16)
[| (oo(ff.)))l & NG val| (
(c) Exist scalars mi and m2 that for all & e N satisfy:
Vlg $ S my + m2/l m
(A17)
Combining assumption 6 and restriction criteria as presented in equation (5), then we have:
E S mi + m / ( with
(A18)
mG m2 + 2 2 ~²>0
Then by substituting Eet [llo((l [)) from equation (A8) into equation (A3), we have:
Be [R( + -
V c())
(A19)
L + oi 2 [m ma/l m
Assumption 5 ensures that {ar) + is practically achievable by applying learning rate sched- uler (with decay) that reduces the learning rate in each step of local training. Then by choosing af Lmag < and substituting V <f)] in equation (A9) with the condition in sumption 6.b, we have
E -0(()))))))))))))))))))))))))- - a
(A20)
#² L + 2 (ml +
Applying expectation into the equation above get
-B(r()))))))) S -f
+ a +21
2 (mm v
(A21)
-
} (1
22 of Lm
Sum both sides for E (1,..., K} we get
For - E(F(6)) s BIA(OK -
K
K
(A22)
Faf Fa < [
+ Lmn
a
62
km
Dividing by for both sides, then get
(
((E[ Fins)
Lmm
K
2
(A23)
k=
17

---

Published as a conference paper at ICLR 2025
Applying timx+xo and assumption 5 to the equation above get
lim
(EIIIA((
2
Fint)
(A24)
L lim K-400
< 00
Dividing both sides with Σ k= af. and following assumption 5 where limk- Σ aq and limx- +00 Ex= at² co, then the right side will return O. Therefore, we have
Σ Baa (F(( B
lim
=0
(A25)
Σ of
ΣK of v |=0
lim R0000
(A26)
Σ of
lim BII =0
(A27)
The equation (A17) proves the convergence for local training in (-th client where the gradient of loss F converges to 0 along with the increase of training step/iteration and the decreasing of learning rate a.
B.2 PROOF OF THEOREM 2
Let the selected local clients (1)= are conduct local optimization with its local training data {T,' U 2) coordinated by central server G, where T is local training sample for client / for task Client local update is conducted
iterations using minibatch sampling on local training data set 5 E 'T" Global model aggregation is executed in each round {1, 2,.,, R). We define global aggregation step Ig {rErr r 2......R) Following (Li et al., 2019), symbol 6/++1 denotes the local parameter of client / after communication steps, while s denotes the local parameter after an immediate result of one step of stochastic gradient descent. Then the definition satisfies the following expressions:
F1
x+1
o at a'Vin(Ff.")
(A28)
9i k- if E Ig Ehe u p
0
(A29)
k+1 if E
Where wi = w// Ets Wt, where w/ is the weight of client We state gK+1 = Ets wight h+1 and 6utt = EIe w/ # is the result of single step of stochastic gradient descent iteration from 6ft1 Then we define g² D/s wi W Fi()) and g² Dhs wi VFFF $²). We adopt the lemmas below from (Li et al., 2019) where their derivation is obtained from fully participating clients in FL setting.
Lemma 1: By applying assumptions and 2, in step SGD training and chose iZ we have
B| 111 < (1 a E) 1+ (a"l"
101131 + 6L(a P +
2 DLs
A=1 wy @ 1|] where r PM EtE wx Fi 2 0.
Lemma 2: By applying assumption 3, the gradient function follows:
Ellgg 3-11 < Els of where of is the variance of θ
Lemma 3: By applying assumption 4, where a is non-increasing and it satisfies a S
k+E all:: 2 0, then have E |@ 0 2(
In FL setting with all clients participating, get e sh-1. However, in setting with only partial clients participating, we utilize a random sampling mechanism that satisfies Es, [6 k+1 We also adopt the bounding condition from (Li et al., 2019) as shown in lemma 4.
18

---

Published as a conference paper at ICLR 2025
Lemma 4: The expected difference between + and g bounded by: Es, |1@ G 4 Ly ( and in the case of wi uniform for all I-th client, then Es, [1@ ( 6 1 -
4(Ns-La) (
where Ns is total clients and Lg is number of selected clients. (
Ny-1
Please note that
|| - 0¹1 | - -111
(A30)
11 - 0-1 = || 5tt + - -
(A31)
|| - 001 = 1 - 11 2 11+11gth1 0-11
(A32)
+ +211 +1 x 11-1 666
|| - -0111 -| - x 12 + ++110++1 0-11
(A33)
2 ++1 x x6-1 18 k+1 0-)
G vanishes. Then by applying lemma 4, we In the case of k+ + 1 $ Ig, then the term ||@k+1 -10k+11@gtt+
get
Bl|l@**1 0*1l²) < (1- a² -011)+ (a*)
+011++++
(A34)
In the case of k + 1 E Ig, then by applying lemma 4, we get
B| - 01| -
< (1 -
(A35)
where B = Eta
L8 E foon + 6LE + 8(E G) and C 4(Ns-Ls) No-1
GE* if is uniform and
= 1/2 GE otherwise.
By choosing a²
B for some B 1/p and 5 0 so that a S min 1/4L] 1/4L and
k+o
a $ "+ then we have [ e 11 S 8+ where maxx (B+C)
Au-1 (8 1)
Then, by applying strong convexity assumption of F we have
B S <
(A36)
2 8 &
where F is the minimum value of F where optimum parameter O* is achieved. Later on, if we choose 3 = 2/11. 5 = max max(8T/p1. an denote K = T./1, a =1/u(1/ (5 + k:) then we have
C μ El e*
E(P(@*)) F < (6+k-1)
(A37)
The equation above generalizes FI. where the model is optimized in a total l: iterations where in practical implementation, k b.E. b the number of batches. Then, we get k > R as and b are positive integers. Therefore, substituting & with R in the inequation above produces the higher amount on the right side. On that basis, the inequation above can be derived into:
2(B + C)
uō Elle'
E[F(""]] - F" R
(A38)
P
2
The equation above can be derived into:
2(B + C) E[F(O")) F R 188/R+1-1/1
nd
E -
(A39)
"
2
K Let A (8/14Ī-1/R) is a positive number. Then the equation above can be derived into:
-] < A
2( n8 Ello -'11)
(A40)
R
P
The inequation (A30) guarantees the proposed weighted federated learning achieves a convergence condition that is upper bounded by the amount on the right side.
19

---

Published as a conference paper at ICLR 2025
B.3 PROOF OF THEOREM 3
Let @r and θ are optimal parameter in T/ U Z and T respectively, where Ti C T where
IT/1/TT" 17 E (0, 1), then we have
T: + (1 F(e T:'))
(A41)
TT) = (1 -T')
(A42)
Let that 0 is the initial value of Oand 0 assigned by random uniform initiation. Thus or all e" After optimally learning Ti' and Ti' U Z then Oo become è and O' respectively. Note that knows only available classes in T,", while O knows classes that available both in T,' and classes in T Ti via Z. Assuming that the loss for predicting classes in T/' is e" e", then we have F(O; TT') F(O": T c² c². Since all the backbone parameters are frozen and O* learn Z, then we get F(O; (T - -T()) c", while F(O"; (T' - T)') = =e", where ea 2 e² 2 e".
class E To Ty It satisfy F(@"; (()):T))
Then, the equations (A41) and (42) can be derived
F(O;T*) = np² + (1 - n)
(A43)
; = ne 7)e*
(A44)
Subtracting the equations above, then we have
-("T))=((0777))) me"" + (1 n)E (me" + (1 - n1)e")
(A45)
- = Ne + (1 1TJE" ne - (1 - n)e
(A46)
-(0;T))-((6 - (T)) = (1 n)e - (1 - n)
(A47)
F(0,T)) F T') = (1 - -) - ed)
(A48)
As we have 0 < < 1 and e" > c², the right side of the inequation above has a positive value.
Then, by choosing small positive number 0 where (1 ()) -ea) then we have.
F(O; T') F(O':T') 2
(A49)
Inequation above proves that @" more generalized to T than θ. This shows that our idea i.e. empowering prompt learning with shared unified prototypes improves model generalization.
C DETAILED COMPELXITY ANALYSIS
Following the pseudo-code in Algorithm 1, UOPP have several operations e.g. generate static proto- type (line 11, 24, 34), drawing S, Q from prototypes (line 25), Rectify prototype (line 26), updating model parameters (line 20-22, 28-30), forming unified prototype (line 27, 38, 40) data exchange between clients and server. Knowing that accumulating on all batches, generating prototype or com- pute features from T/' cost O(N)), drawing (augment) samples from feature costs O(N)), rectifying prototypes cost costs O(N)), parameters update cost costs O(N)), forming uniform prototype cost O(1), and parameters exchange include aggregation costs O(1), and we have base task and T few-shot tasks (total task is (T+1)) then the UOPP complexity will be:
OUUPPP) = 0 ((BaseTask) + O(FewStotTask))
(A50)
O(UOPP) =0(1) (r(((ciientsssee))) + Osserveraan)
(A51)
+0(1) +T.Ry. (O.dr.(O +0 (serveer)))
OUOPP)) =0(1) + (L..(..L.(())e))nnnnaaa...))) + Osserverha.)))
(A52)
+ -T..R--LL.O(dddientt))) + O(server).))
20

---

Published as a conference paper at ICLR 2025
+0(E.N))) (O(EN)))))))))))))))))))))))))))
(A53)
+((.N))+((FEN))+++++++))+
=0()) +4 +T.RI.LO(EEN)
(A54)
O(UOPP) -0......00 +T..T.L.E..(N))
(A55)
(UOOP)=-(1) -,, +T.00()))))
(A56)
Please note that N₁ Np N7 + NY N T(N)), E [1.7]. Therefore, the equation
above can be derived into:
OUUOPP)
(A57)
O(UOPP) = R...O.)(())
(A58)
O(UOPP) O(RT.L.E.N)
(A59)
Since E is set as a small constant in our method i.e. 1-20 and Rr < R, then the UOPP complexity will be:
OUUOP)) O(RLL.)
(A60)
Our derivation shows that our proposed method has the complexity of O(R.L.Ni) where is total global rounds, L is the number of selected local clients in each round and N₃ is the number of samples in each client.
D EXTENDED EXPERIMENT RESULTS AND ANALYSIS
D.1 EVALUATION ON STANDALONE FSCIL
We measure the performance of our proposed method in standalone FSCIL setting to evaluate our idea on prompting with a unified static-dynamic prototype and dual-head classifiers. We compare our method with existing SOTAs iee TEEN(NeurIPS, 2023)
al., 2024), NC- ESCILICLR, 2023)(Yang et al., 2023), OrCo(CVPR, 2024) et al., 2024), and PriVi- Lege(CVPR, 2Pak) et al., 2024). The evaluation is conducted in datasets i.e. CIFAR100, MinilmageNet, and CUB200, following common settings in FSCIL.
Table A6 shows the detailed numerical result of our experiment. The table shows that our proposed method achieves a better performance in general. In comparison to TEEN, NC-FSCIL, and ORCO, our method archives significantly better performance i.c. with more than 20%, 25% and 14% mar- gin in CIFARI00, Minilmagenet, and CUB200 dataset respectively. In comparison to Pri PriVeLedge, Our method achieves higher performance in CIFAR100 and CUB200 dataset with 1.65% and 4.9% margins respectively. Our method achieves lower performance than PriVil in MinilmageNet dataset. Looking in a more derailed view, that lower performance basically due to our method achieving lower performance in the base task i.c. 3% lower accuracy. Thus, It affects the later (few-shot) tasks and average performance. Please note that PriviLedge utilize vision and language modality, while the rest methods only utilize vision modality. This factor should be considered into account in the performance comparison.
Looking from the performance drop (PD) metrics, method archives the lowest performance drop all dataset by significant margin i.c. 28%. This indicates that our method handle the catastrophic forgetting better than the existing methods.
21

---

Published as a conference paper at ICLR 2025
Table A6: Numerical result of the FSCII. methods in CI MinilmageNet and CUB200 dataset with 5-shot setting, PD indicates the performance drop, and Gap indicates the gap between the respected method to our proposed method (UOPP).
Method
Accuracy in each session (%)
Avg
PD
Gap
CIFARIOU: 60 base classes @5 classes in few-shot task
THEN [NeurIPS,2022
74.92
72.65
68.74
65.01
62.01
59.29
57.90
54.76
52.64
63.10
22. 22
26.63
NC-FSCII (ICLR 2023)
89.51
82.62
76.72
71.61
63.18
59.67
56.53
53.70
35.81
20.77
O:CO (CVPR 2024)
80.08
68.16
66.99
60.97
59.78
58.60
57.04
55.13
52.19
62.10
27.89
27.63
PriVil ege (CVPR 2024)
90.88
89.39
88.97
87.55
87.83
87.35
87.53
87.15
86.06
88.08
4.82
1.65
UOPP (Ours)
90.27
90.45
90.14
89.40
89.26
89.52
89.70
89.48
89.35
89.73
0.92
0.00
MinilmageNet:
60
base classes 02.5
classes in few-shot task
TEEN (NeurIPS.2022
73.53
70.55
66.37
63,23
60.53
57.95
55.24
53.44
$2.08
61.44
21.45
31.58
NC. FCCTSCIL.C (ICLR 2023)
77.25
71.30
66.21
61.80
57.94
54.53
51.50
48.79
46.35
59.52
30.90
33.50
0:CO (CVPR 2024)
83.30
75.32
71.53
68. 16
65.63
63.12
60.20
58.82
58.08
67.13
25 25.22
25.89
PriViLege (CVPR 2024)
96.49
95.54
94.91
95.27
2.58
-2.25
UOPP (Ours)
93.57
93.66
93.51
93.51
93.13
92.24
92.23
92.66
92.71
93.02
0.86
0.00
CUB200:
100 buse classes
10 clusses in few-shol lask
THEN (NeurIPS 2023)
17.26
76.13
72.81
68.16
64:40
62.29
61.19
60.32
59.31
14.14
17.95
14.82
NC-FSCIL (ICLR 2023)
78.49
71 52
65.54
60.30
55.81
51.96
48.72
45. 78
43.
40.92
38.80
57.92
39.69
25.04
O:CO (CVPR 2024)
75.59
66.85
64.05
63.69
62.20
60.38
60.18
59.20 29.20
58.00
58.42
57.94
63.35
17.66
19.61
PriViLege (CVPR 2024)
82.21
81.25
80.45
77.76
77.78
75.95
75.69
76.00
75.19
75.19
75.08
78.03
7.13
4.93
UOPP (Ours)
86.63
86.99
85.19
83.37
82.42
80.19
80.21
80.76
80.93
81.16
81.17
82.96
5,46
0.00
D.2 PERFORMANCE ON DIFFERENT K-SHOT
We extend our investigation of the methods' performance different k-shot values i.e. 7-shot, 5-shot, 3-shot, and I-shot. Table A7 presents the detailed numerical results on CIFAR100 dataset with 1-7 shot settings. Table A7 shows that our proposed method consistently achieves the high- est performance in different of k-shot values. The performance gap is consistently significant i.e. 7-46%. Regarding average performance, our method tends achieve lower performance in lower k-shot values i.c. and 3. UOPP performances in 5-shot and 7-setting are comparable on average. However, in terms of the performance in the last (final) session, UOPP achieves better final per- formance in a 7-shot setting. This indicates that more samples for each client improve the global model performance.
This trend also applies to other methods where their performance on the higher shots is better than their performance on the lower shots. However, Fed-CPrompt shows the anomaly, where its perfor- mance in lower shots is better than its performance in higher shots. This condition indicates that the increase in novel classes' accuracy is lower than the drop in base classes' accuracy. Thus, is higher sample is contra- produccvee for few-shot task training. Looking to the performance drop (PD), the table shows that UOPP achieves a lower (better) performance drop in the higher k-shot value. This fact is in line with the mentioned trend of higher performance in the higher k-shot value before.
D.3 MEMORY CONSUMPTION ANALYSIS
We extended our resource analysis by evaluating the memory consumption by client and server. Table A8 shows the memory consumption by a single client and central server. Please note that each client trains its local model utilizing its local dataset, while the central server coordinates the clients, aggregates the locally optimized models into global model, and redistributes it to the clients.
Table A8 shows the memory consumption for our proposed method, Fed-DualP, and PILoRA where these 3 methods have comparable communication costs and similar aggregation methods. As com- monly known, Table A8 shows the memory consumption of deep learning training highly affected by the batch size (BS) of the data loader, while the memory consumption for model aggregation affected by the number of participating clients. Second, the table shows that the memory con- sumption for a central server is relatively lower than the client's memory consumption. Third, the memory consumption of our method comparable with Fed-DualP, and PILoRa even though our method utilizes more operation for prototype rectification and has more trainable parameters dur- ing the few-shot tasks. This fact indicates that our method has comparable scalability and resource requirements with the existing methods. Last but not least, the table shows the high possibility of practical deployment of our method both in cross-silo and cross-device scenarios. The local training setting can be adjusted by the end device specification. FOr example, in the case of cross-device
22

---

Published as a conference paper at ICLR 2025
Table A7: Numerical result of the FFSCIL methods in CIFAR1O0, 7-shot, 5-shot, 3-shot, and 1-shot setting, indicate number shot, indicates the performance drop, and Gap indicates the gap between the respected method to our proposed method (UOPP).
Method
Accuracy in each session (%)
Avg
PD
Gap
Fed-S3C
43.98
49.65
48.01
45.83
43.21
41.68
40.39
39.69
38.18
43.40
5.80
46.36
TARGET
68.65
63.40
58.36
52.68
50.04
46.58
43.96
39.37
34.04
50.78
34.61
38.98
LGA
73.15
67.08
64.56
59.87
56.15
52.04
50.00
47.29
44.39
57.17
76
32.59
LANDER
66.30
60.65
55.89
$1.05
48. 18.15
44.53
42.66
40.83
38.
49.81
28.04
39 95
Fed DualP
Fed-Cprompt
88.13
77.9385
69 6857
67.13
66,7125
64 64,8824
64.28
62.51
61.51
69.20
26.62
20.56
PILORA
66.48
61.37
56,99
53.19
49.86
46.93
44.32
41.99
39.89
51.22
26.59
38.
UOPP
90,77
90.86
91.04
90.75
91.08
89.35
88.84
87.87
87.32
89.76
3.45
0,00
Fed-S3C
44.31
48.97
47.77
45.35
43.48
41.47
40.33
39.32
37.71
43.21
0.80
46.80
TARGET
LGA
5
73.76
69.80
65.59
60.26
$6.87
$2.94
50,66
47.69
44.89
$8.05
28.87
31.96
LANDER
58.60
61.75
56.26
52.11
47.71
44.71
41.69
40.28
38.87
49.11
19.73
40.90
Fed LeP
73.47
74.20
73,37
71.88
70.85
70.72
69.28
68.66
68.37
71.20
5,10
18.81
DualP
76.39
82.75
83.37
80.80
79.93
78.26
77.73
76.98
77.11
79.26
-0.72
10.75
Fed-CODAP
81,73
70.81
68.67
67.17
14
64.32
64.12
68.56
17.62
21.45
Fed-Cprompt
88.00
64.63
69.30
67.39
63.39
62.33
61.11
59.78
59.00
66.10
29.00
23.91
PILoRA
53.92
TOPP
90.57
90.58
90.85
90.96
91.23
91.51
91.56
91.74
$1.05
90101
9.52
0.00
Fod-S3C
43.98
49.60
48.03
45.19
69
41.12
39.87
38.82
37.47
42.97
6.51
45.64
TARGET
63.03
58,10
52.64
49.00
44.92
32,20
49.76
38.85
LGA
73.52
67.37
63.57
$9.07
$6.01
$2.35
49.50
46.89
44.15
56.94
29.37
31.67
LANDER
59.77
Fed- DualP
73.20
79.31
82,73
81.53
82.06
81.16
81.56
81.21
80.34
80.34
-7.14
8.27
Fed-Cprompt
88.32
83.62
81,46
77.68
77.18
74.32
68.51
77.46
19.81
11.15
PILORA
66.43
61,32
56.94
53.15
29.83
46.89
44.29
41.96
39.86
51.19
26.57
37.42
UOPP
90.62
90.57
90.84
89.84
90.18
89.69
88.20
88.24
79.31
$8.61
11.31
0.00
TARGET
68.90
63.61
59.06
55. 12
51 668
48. 64
45.94
43.52
41.34
53.09
27.56
35.53
LGA
73.58
67.00
63.44
58.88
56.90
54.00
52.82
49.25
48.78
58.29
24.80
30.35
LANDER
57.60
61.75
56.09
$1.29
47.33
44.22
39.73
40.15
37.70
48.43
19.90
40.19
Fed 1.2P
77.00
74.91
74.73
74.40
73.45
74.20
73.22
73.46
73.25
74.29
3.75
14.33
Fed-DualP
Fed CODAP
83.29
73.27
72.54
68.37
67.52
65.54
62.11
64.19
61.67
68.72
21.62
19.90
Cprompt
79.24
73.82
PILORA
67.20
62.03
57.60
53,76
50.40
47.44
44,80
42.44
40.32
51.78
26.88
36.84
UOPP
90.65
90.16
89.97
89.49
89.53
S8.29
87.66
87.04
84.82
88.62
5.83
0.00
23

---

Published as a conference paper at ICLR 2025
Table A8: Analysis of Memory Consumptions Client and Server
Single Client Memory Consumplion
Method
BS=128
BS-64
BS=32
BS=16
BS=8
BS-4
Fed-DualP
14.88
9.15
5.82
4.2
3.25
2.82
PILoRA
15.13
9.7
7.02
4.33
3.6
3.2
UOPP
14.89
8.68
5.59
3.98
3,26
2,83
Server Memory Consumption (GB)
Method
L=1000
L=100
L=50
L=10
L=5
L=2
Fed-Duall
1.32
0.1320
0,0660
0.0013
0.0066
0.00126
PILaRA
UOPP
1.63
0,1627
0.0814
0.0016
0.0081
0.0033
Table A9: Running Time w.r.t. Percentage of Available Classes Number of Selected Local Client Fach
Round
Run time (h) in different available classes percentage (n))
Run time (h) im different selected local clients (L)
Method
0=0.6600%)
x=0,4(40%
y=00,2/20%
L=4
L=6
L=8
Fed-S3C
4.10
2.07
2.08
2.18
4.10
6.23
TARGET
2.02
2.15
1.99
1.50
2,02
2.50
8.22
LANDER
4.78
1.90
1.85
3.90
4.78
4.60
Fed-DualP
3.20
2.17
1.90
2.07
3.20
4.07
Fed Cpompt
2.03
2.37
2.45
1.50
2.03
2.77
PII
6.17
4,78
3.13
4.12
6.17
8.62
UOPP
6.05
5.35
4.70
6.05
settings where local training is conducted by small edge devices such as laptops or IoT nodes, the batch size can be set to smaller value i.e. 16 or less. In the case of a cross-silo setting, where local training is conducted by more powerful end devices such corporate server, the batch size can be chosen by the larger value i.c. 128 or more.
D.4 RUNNING TIME ANALYSIS
We extend our scalability analysis by investigating the simulation time w.r.t. the training data size e.g. indicated by the available classes percentage (7)), and the number of participating local clients (L). table A9 shows the running time of our simulation w.r.t. those 2 factors. Please note that our simulation is run on single GPU device. Thus each local training is executed sequentially, Thus, the local training is conducted in parallel way, the difference in the simulation time will be smaller.
The table shows that the ratio of the training data doesn't imply the same running time ratio. For example in the case of 77 0.6 where each client carries 3x the number of samples of 0.2, the running time in those 7 = 0.6 is not equal to 3x training time of 1) = 0.2. In our method, the increase of running time in those case is less than 20% (5.35 to 6.05). Thus, of training samples with r factor will impact far less than r training time. Second, as we mentioned earlier regarding the sequential process of local training, table A9 shows linear trend which is logical result. For example, for UOPP and LORA, the total simulation time can be formulated by 1h. In the real application, local training should be conducted parallelly since each client runs on its local device. The bottleneck will be shifted to the server that receives the locally trained models, and the delay for round-trip communication (model transfer) between clients and server.
found that the scale-up ratio
E DISCUSSION ON LIMITATION AND POTENTIAL SOLUTION
Our study has several limitations that can be improved in future studies.
a) Same v) for each client : First, In our simulation, each client has the same non-i.i.d level repre- sented by the same percentage of available classes (7)). In the real application, each client may have a different degree of class availability from the other client. Thus, the future study, the simulation be extended into variation percentage of available classes where each client is assigned with random (picked from a min-max) r range. This variable raises a new challenge for FFSCIL aside from simulating more realistic setting.
b) Simulation on a single device (GPU): : Second, Our simulation is conducted in a single GPU, where each local training is executed sequentially one by one. This limitation will produce linearly
24

---

Published as a conference paper at ICLR 2025
increasing training time with the number of selected local clients. is less realistic especially when we want to measure the training time. This limitation can be solved by utiliziig a server/workstatee that has multiple GPUs such Nvidia DGX server. The other solution is utilizing multiple cloud devices/servers as the clients.
In this study, our method utilized fixed-size prompt for all clients. Related to the randomly selected available class and the new challenge of different (point a), It will be more realistic if client decides its prompt size following the condition of its local data. The evolving prompt approaches such as ConyPrompt(Rog et al., 2024) and EvoPrompt( al., 2024) may be suitable for those case. However, It raises a new challenge in the aggregation process i.c. how to produce an optimum global model from the different-sizer local models which optimum for the current task as well as previously learned tasks.
Fixed-size of Prompt:
The current version of our method doesn't utilize advanced overfitting handling. Thus, in some cases e.g. in the CIFARI00 dataset, the model may suffer from overfitting indicated by the performance drop in the last task. One of the potential solutions is by applying early stopping during the training process. The other potential solution is applying learning decay to reduce the learning rate in line with the increasing training epochs.
d) Overfitting Handling:
e) Multi-Modality The current version of our proposed method utilizes vision modality only. In Singuage-guided approach may become a prospective
the advance of Vision-Language Models, approach to improve the model performance. Alternatively, language embedding can be utilized for prototype rectification instead of the prototypes generated by VIT.
F EXPERIMENT DETAILS AND HYPER PARAMETER SETTING
Experimental Details: our numerical study executed under a single NVIDIA A100 GPU with 40 GB memory across 3 runs with different random seeds. Fed-L.2P, Fed-DualP, Fed-CODAP, and UOPP train T' number of prompts P E R257665 and & E C[ and head layer 4 while the competitors train whole CNN models following their original implementation. Following (Dong et al.. 2023), each experiment is simulated by 20 total clients and 1 global server, where in each round, (30%) local clients are selected randomly. Each client randomly receives 60% (17 = 0.6) classes. The total global round is set to 90 (10 rounds per class) for CIFARION 100 and MinilmageNet and 110 for CUB200. For all methods, the local training on each client is set with a maximum of 20 epochs, and the learning rate is set by choosing the best value from {0.001, 5.0) by grid search with 2 incremental factors. Our setting is different to the recent study (Jiang et al., 2024), since follows FCIL setting, while our setting follows FSCIL setting for the number of tasks, base classes, and novel classes in the few-shot tasks.
Performance Metric: each session, evaluate the consolidated algorithms to all learned classes with accuracy metrics (Acc()). Besides, we also measure the accuracy of base classes, novel classes and harmonic mean accuracy that indicates the balance between the performance of base classes and novel classes, in other words, it represents stability-plasticiyy performance. We also measure performance drop (PD), the accuracy difference between the first task, and the last task.
CNN-Based Methods: The competitor methods i.c. Fed-S3C LGA, TARGET, and LANDER run with 2-20 local epochs on each client. The learning rate is set with the best result from 0.001 to 5.0 by 5 or 10 increment factor. The other hyperparameters such as weight decay, momentum, and dropout rate are set with their original setting. The methods utilize ResNet18 as the backbone model. LGA utilizes LeNet as the perturbation model. TARGET and LANDER use CNN as their synthesizer model. TARGET and LANDER generate 10000-50000 synthetic images on each task.
Prompt-Based Methods: PILORA and the prompt-based methods i.e. Fed-L2P, Fed-DualP, Fed- CODAP, Fed-CPrompt, and UOPP, are run with ViT backbone. The base task is run with 1-2 epochs, while the few-shot task is run with 2-20 local epochs. For UOPP, the rectification step M is set to 40 steps per iteration. The initial learning rate is set with the best result from 0.001 to 0.2 by a 2 or 5 increment factor. The learning rate for the FS task and base task may be different. The prompt length is set to 5. The dual-head selection is executed in a bateh-wise manner for convenience in implementation. The other parameters are set with the default settings.
25

---

Published as a conference paper at ICLR 2025
G EXTENDED LITERATURE STUDY
Few Shot Class Incremental Learning (FSCIL); Previous studies on FSCIL have attempted to maintain cability-plasticce tradeoff in few labeled sequences of tasks by adding extra representa- tion e.g. TOPIC (Tao et al., 2020b) introduces Neural gas as the graph of mapped features and CEC (Zhang et al., 2021) continually evolves its classifier to adapt new tasks. Another approach modifies its learning mechanism e.g. FSLL (Mazumder et al., 2021) takes a partial parameter of the model be updated with self- supervveee loss, F2M (Shi et al., 2021) finds flat minima regions on the base task then forces parameter update on few shot tasks to reside within the flat region, S3C (Kalla & Biswas, 2022) trains scholastic classifier with supervised loss and MgSvF (Zhao al., 2024) applies multi grained fast-slow learning mechanism. FSCIL methods demonstrate that representa- inference tends to be more stable (less forgetting) than linear classifiers under the data scarcity constraint. Nevertheless, the prototypes have to be still refined to avoid the prototype bias problem due to the data scarcity issue.
tion or prototypes-bagt
Class Incremental Learning (CIL): (Wang et al., 2022b), DualP (Wang al., 2022a), CODA- P (Smith et al., 2023) offer a breakthrough solution for CIL by training small-sized task-wise pa- rameters called prompts while the feature extractor e.g. VIT that contains the biggest parameter numbers stays frozen. It solves task interference because each task has a specific prompt parameter train based on trainable matching key with its sample. This approach simplifies the training process and reduces memory consumption. The prompt-based approach is proven to be more effec- tive than the rehearsal approach e.g. ICARL (Rebuffi al., 2017), EEIL, (Castro et al., 2018). GD (Prabhu et al., 2020). DER++ (Buzzega et al., 2020), that saves exemplars from the previous tasks and replays them along with current task samples, the bias correction approach e-g. BiC (Wu al., 2019) and LUCIR (Hou et al., 2019) that trains an additional task-wis bias layer to balance the model's s stability-plasticity dilemma, and the regularization approach e.g. EWC (Kirkpatrick et al., 2017), MAS (Aljundi et al., 2018), LWF (Li & Hoiem, 2017), and DMC (Zhang et al., 2020) that tunes the base learner parameters to accommodate the previous task and current task. Regardless of its excellent performance in CIL, the prompt- based approach has not yet been proven in federated or few sample settings.
Few Shot Learning (FSL): FSL method e.g. metric learning(Ge, 2018), prototype network (Laenen & Bertinetto, 2021), and Neural ODE (Chen et al., 2018; Zhang et al., 2022) works effectively with few labeled training samples in single session but not yet tested in continual or federated setting. However, it confirms that optimizing the prototypes tackles prototype bias that improves model performance greatly.
Prototype-based Methods: Prototype-based FSCIL method such as TEEN (Wang et al., 2024), NC-FSCIL (Yang al., 2023), and OrCO(Ahmed et al., 2024) shows an insight the importance of prototype adjustment. TEEN recalibrates the prototypes using a similarity ratio between a calibrated prototype to base classes prototype and a novel classes prototype. NC-FSCIL utilizes the neural col- lapse principle for prototype alignment, while OrCo generates multi-angle prototypes to improve class representation and discrimination. Proroype-based FL. methods such as FedPCL.(Tan et al., 2022) and FedNCM(Legate et al., 2024) show how to conduct a prototype learning in federated way, where set of clients coordinated by central server work together to achieve globally optimal prototypes. Prototype-based based FCIL methods such PILoRA show how a learnable prototype im- proves parameter-effieien fine-tuning method to handle catastrophic forgetting with data privacy constrains.
The strengths and weaknesses of CIL, FSL, FSCIL, and FCIL methods above inspire us to tackle the FFSCIL problem by developing rehearsal-free prompt learning method combined with optimal prototypes to minimize communication costs.
H DETAILED NUMERICAL RESULTS ON BENCHMARK DATASETS
In this section, we present the detailed numerical result shown in Tables A10, and A11,-
26

---

Published as a conference paper at ICLR 2025
Table A10: Numerical result of the consolidated algorithms in MinilmageNet dataset with 5-shot and 1-shot setting across different seeded runs. indicates the number of shots for the few shot tasks, PD indicates the performance drop, and Gap indicates the gap between the respected method to our proposed method (UOPP).
Method
S
Accuracy in each session (%)
Avg
PD
Ped-S3C
31.91
32.97
31.74
30.96
29.93
28.92
27.66
27.06
26.43
29.73
5.5
63.2
TARGET
58.10
53.64
49.80
46.48
43.58
41.02
38.74
36.70
34.85
44.77
23.2
48.2
LGA
50.68
49.20
45.19
38.05
29.24
29.91
27.26
25.94
20.17
35.07
30.5
57.8
Fed-L2P
81.02
78.22
78.66
79.44
79.67
78.14
77.65
77.48
80.00
78.92
1.0
14.0
DualP
Fed-CODAP
90.21
83.35
82.31
80.27
78.89
77.79
77.13
75.94
75.09
80. 11
15.1
12.8
Fod-Cprompt
93.57
92.26
90.71
89.60
89.09
87.22
85.80
85.41
85.28 28
88.77
8.29
UOPP
93.65
93.24
92.97
92.60
92.73
92.92
92.49
92.73
92.92
92.92
Fod-S3C
32,84
33.19
31.83
30.69
29.17
27.90
20.54
25.68 66
24.60
29.16
8.2
63.8
49.80
LGA
50.15
41.26
37.24
33.33
26.04
27.02
24.61
23.53
21.71
31,65
28.44
60.56
Fod-L2P
83.02
79.99
79.92
79.54
80.20
80.84
80.55
80.60
82.57
80.80
12.1
Fed DualDDalP
85.47
90.06
88.77
88.44
88.14
86. 22
85.04
84.98
84.80
86.88
Fed-CO Cel-CODAP
90.94
83.65
81.53
80.21
79.50
77.48
76.71
75.89
75.24
80. 13
15.7
12.8
Fod-Cprompt
93.42
91.72
88.94
87.89
87.00
81,82
81.15
81.04
34
12.38
UOPP
93.66
93.15
92.72
92.04
92.20
92.05
91.03
91.56
91.48
92.21
2.2
0.0
Table A11: Numerical result of the consolidated algorithms in CUB200 dataset with 5-shot and 1-sot setting across 3 different seeded runs. S indicates the number of shots for the few shot tasks, PD indicates the performance drop, and Gap indicates the gap between the respected method to our proposed method (UOPP).
Accuracy in each session (%)
Method
S
Avg
PD
Gap
Fed-S3C
18.65
18.54
17.9
13.85
13.41
14.26
13.70
13.19
12.70
12.27
11.43
14.91
1.22
65.89
TARGET
32.03
27.75
25.44
23.48
21.80
20.35
19.08
17.96
16.96
16.07
15.26
21.47
16.77
59.33
LGA
25.07
22.95
19.89
17.86
16.
14.94
13.88
12.26
11.19
10.91
16.92
63.88
Fod-L2P
73.24
69.49
63.32
60.46
61.05
56.65
53.24
51.98
51.47
49.42
50.57
58.26
22.67
22.54
Fed-DualP
Fed. COD-CODAP
71.69
53.03
42.26
32.81
34.38
29.69
30.73
30,10
29.24
29.73
29 .4.44
37.55
42.26
43.25
Fed-CPrompt
82.02
60.76
59,24
52.15
50.76
50.77
50.53
50.48
61.23
19.57
UOPP
86.18
85.95
84.96
83.02
81.62
79.48
78.57
78.15
77.70
77.86
75.28
80.80
10.90
0.00
Fed-S3C
18.65
18.22
17.34
15.65
14.86
13.64
13.22
12.64
11.80
11.57
10.79
14,40
7.85
62.33
27.01
L.GAA
23.87
10.74
10.67
10.17
8.83
9.55
7.71
8.88
7.25
6.55
6.01
10.02
17.86
66.71
Fed-L22
73.74
67.78
62.05
58.91
61.08
57.03
52.70
48.41
46.96
49.40
57.12
24.34
19.61
Fed-DualP
78.20
76.41
71.23
66.34
63.63
62.69
58.37
54.25
51.67
50.48
51.27
62.41
26.94
14.32
Fed CODAP
73.07
56.54
48.81
39.62
37.46
35.15
32.56
30.81
28.32
28.10
27.39
39.80
45.68
36.93
Ped-CPromgl
87.22
72.41
66.88
50.86
59.63
53.31
51.58
50.99
47.60
50.02
50.36
58.26
36.86
18.47
UOPP
85.88
84.66
83.17
80.19
79.19
76.57
74.02
72.71
71.05
70.26
66.34
76.73
19.54
0.00
27

---

Published as a conference paper at ICLR 2025
Table A12: Base classes accuracy of the consolidated algorithms in CIFARI00 dataset with 5-shot and I-shot setting across 3 different seeded runs. S indicates the number of shots for the few shot tasks, PD indicates the performance drop, and Gap indicates the gap between the respected method to our proposed method (UOPP).
Method
Base Classes Accuracy int each session (%)
Avg
Capp
6
53C
44.51
-3.07
TARGET
5
68.91
68.91
68.91
68.91
68.91
68.91
68.91
68.91
68.91
68.91
0.00
19.70
LGA
73.76
72.92
73.17
72.55
72.79
72.
72.62
72.40
72.07
72.71
1.36
15.90
LANDER
66.90
65.63
65,13
63.62
63.33
62.53
63.78
64.78
64.15
64.43
2.12
24.18
Fed L2P
73.47
77.66
79.49
81.02
82.53
83.81
83.53
84.19
84.78
81.17
-10722
7.44
Fed-DualP
76.39
85.00
87.19
87.50
87.85
87.60
87.98
86
88.09
86.17
-11.47
2.44
Fod-CODAP
81.73
69.20
71.38
70.26
68.93
69. 48
68.23
70.16
70.88
71.14
11.58
17:47
Fed-Cprompt
63.25
22.74
UOPP
90.57
90.57
90,56
90.57
90.57
90.56
90.57
90.56 56
72.97
88,61
0.01
0.00
Fed-S3C
44.51
49.42
49.96
49.52
19.56
49.21
48.97
48.93 93
48. 06
48,6 68.68
-4,42
40.68
TARGET
68.91
68.91
68.91
68.91
68.91
68.91
68.91
0,00
20.46
LGA
73.58
72.41
73.57
72.26
72.63
71.87
72:40
70.34
70.05
72.12
3.24
17.24
62.65
Fed 1.2
77.00
78.36
79.67
80.19
80.59
81.52
81.00
80.93
80.33
79.95
-3.93
9.41
Fod-DualP
78.66
86.60
88.35
87.74
87.82
87.88
87.57
87.70
87.11
86.60
-9.04
2.76
Fed CODAP
83.29
74.56
74.18
72.00
71.12
69.91
67.09
70.44
68.42
72.33
12.85
17.03
Fed-Cprompt
87.53
85.38
82.57
40
80.73
79.60 99.60
79.48
77.92
75.77
81.15
9.62
8.21
UOPP
90.65
90.65
90.65
90.66
90.66
89.48
88.82
87.97
84.72
2.68
0.00
Table A13: Novel classes accuracy the consolidated algorithms in CIFAR100 dataset with 5-shot and 1-shot setting across 3 different seeded runs. S indicates the number of shots for the few shot tasks, PD indicates the performance drop, and Gap indicates the gap between the respected method our proposed method (UOPP).
Method
Accuracy sessson
Avg
Gup
Fod-S3C
49.80
37.67
31.00
27.85
25.31
25.70
25.16
24.48
31.00
25.32
61.92
TARGET
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
92.92
LGA
32.27
20.10
11.13
9.12
6.87
6.75
5.33
4.13
11.96
28.14
80.96
LANDER
Fed-L.2P
32.67
36.67
35.29
35.82
39.29
40.77
42,04
43.76
38. 29
-11.09
$4.63
55.80
57.
Fed-CODAP
70:40
67.40
62.33
61.88
58.12
56.52
55.60
53.98
60.78
16.43
32.14
Fed-Cprompt
88.20
83.70
68.07
63.80
62.20
$8.37
$8.23
58.58
67.64
29.63
25,27
UOPP
90.73
92.60
92.56
93,20
93.77
93.53
93.75
93.18
92.92
-2.55
0.00
Fed-S3C
40.07
27.53
23.22
19.67
18:27
17.60
17.49
16.52
22.55
23.55
62.74
TARGET
0.00
0.00
0.00
0.00
85.29
LGA
2.07
2.67
5,36
9.68
11.11
13.67
13.11
16.88
9.32
-14.81
75.97
LANDER
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
85,29
1.2P
33.47
45.10
$1,24
$2.02
56.65
$7.67
60.66
62.63
52.43
29.166
32,86
Fed-DualP
65.87
67.07
67.02
66.85
66.48
67.98
67.46
68.15
67.11
-2.28
18.18
Fed-Cprompt
52.60
59.30
48.73
49.95
46.68
45.87
45.06
43.45
48.95
9.15
36.34
UOPP
84.27
85.90
84.82
86.15
85.34
85,46
84.96
85.29
-0.69
0.00
I DETAILED NUMERICAL RESULTS ON STABILITY-PLATTICN ANALYSIS
In this section, we present the detailed numerical results on the cability-plaaticiig analysis of UOPP as shown in Tables A12, A13, and A14.
J DETAILED NUMERICAL RESULTS DIFFERENT LOCAL CLIENTS AND GLOBAL ROUNDS
In this section we present the detailed numerical results on different local clients and rounds as presented in tables A15 and A16.
K DETAILED NUMERICAL RESULTS OF ABLATION STUDY
In this section we present detailed numerical results the ablation study as shown in table A17.
28

---

Published as a conference paper at ICLR 2025
Table A14: harmonic Mean accuracy of the consolidated algorithms in CIFAR100 dataset with 5- shot and I-shot setting across different seeded runs. S indicates the number of shots for the few shot tasks, PD indicates the performance drop, and Gap indicates the gap between the respected method to our proposed method (UOPP).
Harmonic Mean Accuracy each session (%)
Method
Avg
PD
Gap
Fed-S3C
49.35
42.76
37.95
35.45
33.93
33.39
32.91
32.08
37.23
17.20
53.27
TARGET
5
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
90,50
5
44.74
19.30
16.20
12.54
12.35
36.93
71.20
LANDER
5
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
90.50
Fed-L2P
5
45.99
50.18
49.16
49.95
53.50
$4.79
56.U8
57.72
52.17
-11.74
38.33
Fed DualP
67.37
71.39
66.80
68.44
68.20
69.34
70.12
71,83
69.19
4.46
21.31
Fed-CODAP
5
69.79
69.33 33
66.00
65.21
63.30
61.82
62.04
61.28
64.85
8.51
25.65
Fed-Cprompt
73.27
74.36
64
63.52
62.29
60.35
$8.93
14.35
25.52
oRL
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
90.50
UOPP
90.65
57
91.55
86
92.14
92.03
92.13
81,85
90.47
8.80
0.00
Fed-S3C
44.25
35.50
31.62
28.16
26.64
25.90
25.77
24.58
30.30
19.67
56,90
TARGET
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
87,20
4.02
-23.19
71.23
LANDER
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
87.20
46.90
Fed-DualP
74.82
76.25
76.00
75.91
75.70
76.54
76.26
76.47
75.99
-1,65
11.21
Fed CODAP
65.12
67.93
61.63
63.11
61.59
58.67
60.80
58.80
62.21
6.32
24.99
Fed-Cprompt
65.10
69.03
60.97
61.72
58.85
58.17
57.10
55.23
60.77
9.87
26.43
PILoRA
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
87,20
LOPP
Table A15: Accuracy of the consolidated algorithms in CIFAR100 dataset with 5-shot setting on different number of selected local clients across different seeded runs. S indicatess the number of shots for the few shot tasks, PD indicates the performance drop, and L. indicates the number of selected local clients.
Accuracy in each session (%)
Meihod
Avg
40.39
38.39
43.94
S3C
44.51
48.97
47.77
45.35
43.48
41.47
40.33
39.32
37.71
43.21
6,80
38.
37.21
42.74
TARGET
66.75
61.62
57.21
53.40
50.06
47.12
44.50
42.16
40.05
51.43
26.70
TARGET
90
63.61
59.06
55.12
51.68
48.64
45.9 45.94
43.52
41.34
$3.09
27.56
TARGET
49.02
56.66
LGA
72.98
68.8
62.81
57.57
54.29
51.31
49.81
46.67
42.58
56.34
30.40
LGA
73.76
69.80
65.59
60.26
56.87
$2.94
50.66
47.69
44.89
58.05
28.87
LGA
73.73
70.03
60.4
56.31
52.68
50.37
47.32
44.61
57.93
29.12
LANDER
59.60
$8.03
53.23
49.81
45.80
43.11
40.64
39.06
37.27
47.40
22.33
LANDER
60
61.75
52.11
47.71
44.71
41.60
40.28
38.87
49.11
19.73
LANDER
61.60
63.80
58.84
54.37
50.46
47.76
44.17
42.18
40.82
51.56
20.78
Fod-DualP
64.90
75.14
80.11
78.63
79.35
77.79
76.84
76.67
76.05
76.17
-11.15
Fed-DualP
77.73
76.98
79.26
Fed- DualP
84.65
84,77
83.90
80.41
78.56
76.64
75.93
74.60
73.58
79.23
11.07
Fod-Cprompe
87.78
44,88
35.84
35.13
38.63
40.78
38.12
41.14
41.30
44.84
46.48
Fed Cprompe
88. 00
64.63
69.30
67.39
63.39
62.33
61.11
59.78
59.00
66.10
29.00
Fed-Cprompt
87.65
82.52
80.99
77.53
76.64
73.74
71.70
70.67
68.39
76.65
19.26
89.06
UOPP
90.57
90.58
90.85
90.96
91.23
91.51
91.56
91.74
81.05
90.01
9.52
UOPP
90.93
90.91
91.40
91.61
91.74
91.78
91.26
91.36
90.90
91.32
0.03
29

---

Published as a conference paper at ICLR 2025
Table A16: Accuracy of the consolidated algorithms in CIFARIOO dataset with 5-shot setting on different number of rounds across 3 different seeded runs S indicates the number of shots for the few shot tasks, PD indicates the performance drop, and R indicates the number of rounds.
Method
R
Accuracy in each session (%)
Avg
PD
S3C
54
43.42
49.51
48.01
45.41
43.69
41.96
40.89
39.64
38.
43,40
5.32
S3C
72
$1.20
53.95
51.97
49.09
47.10
45.11
43.74
42.88
41.13
47.35
10.07
S3C
90
44,51
48,97
47.77
45.35
43.48
41.47
40.33
39.32
37.71
43,21
6.80
TARGET
34
$7.60
53.17
49.37
46.08
43.20
40.66
38.40
36.38
34.56
44.38
23.04
TARGET
72
51.84
TARGET
90
68.90
63.61
59.06
55.12
51.68
48.64
45.94
43.52
41.34
$3.09
27. 36.56
LGA
54
69.57
62.32
60.94
61.41
57.44
53.71
49.76
51.24
47.51
57.10
22.06
LGA
72
45.19
42.08
55.24
26.27
LGA
90
73.76
69.80
65.59
60.26
56.87
52.94
50.66
47.69
44.89
$8.05
28.87
LANDER
54
60.60
43.89
40.59
38.16
34,89
32.74
31.04
30.14
29.23
37.92
31.37
LANDER
72
62.60
57.00
52.09
49.21
46.21
43.13
40.42
37.05
36.64
47.15
25.96
LANDER
90
58.60
61.75
56.26
52.11
47.71
44.71
41.69
40.28
38.87
49.11
19.73
Fed-DualP
-0.92
Fed-DualP
72
78.85
83,25
83.96
81.61
80,28
79.73
78.63
78.15
77.06
80.17
1.79
Fod-DaalP
90
76.39
82.75
83.37
80.80
79.93
78.26
77.73
76.98
77.11
79.26
-0.72
Fed-Cprompt
34
72,05
64.70
54,99
52.84
36.98
55.86
51.01
62.49
36.91
Fed-Cprompt
72
87.92
72.42
49.91
57.41
60.11
55.87
53.98
49.23
49.84
59.63
38.08
Fed-Cprumpi
90
63,39
62.33
59.78
59.00
UOPP
54
89.87
89.75
90.33
90.77
90.91
91.12
91.33
91.18
91.32
90.75
-1.45
UOPP
72
90.37
90.29
90.70
90.99
90.94
91.34
91.27
91.07
90.13
90.79
0.24
UOPP
90
91.23
91.51
91.56
91.74
81.05
90.01
9.52
Table A17: Accuracy of different configurations in CIFAR100 dataset with 5-shot setting across 3 different seeded runs. S indicates the number of shots for the few shot tasks, PD indicates the performance drop, and Gap indicates the difference accuracy PIP.
Conf.
Accuracy
(%)
Avg
PD
Gap
A (wio Static Proto)
84.37
82.54
80.91
80.56
80.29
80.54
80.70
79.03
76.61
80.62
7.76
9.39
Dynamic Protu)
C. (wro MI.P Head)
88.25
88.66
89.07
89.16
89.63
90.11
90.27
90.62
82.76
88.72
5.49
1.29
(Wl Head)
90.10
83,
77.23
72.08
67.58
52.34
69.23
37.76
20.78
UOPP
90.57
90.58
90.85
90.96
91.23
91.51
91.56
91.74
81.05
90.01
9.52
0.00
30


Reviews:
[{"review_id": "efJL4ybqlS", "reviewer": "Reviewer_KvkJ", "summary": "The paper introduces a novel model named Unified Optimized Prototype Prompt (UOPP) to address the Federated Few-Shot Class-Incremental Learning (FFSCIL) problem. This problem is characterized by clients holding very few samples for new classes, which presents challenges such as catastrophic forgetting, overfitting, and prototype bias. The UOPP model incorporates task-wise prompt learning, unified static-dynamic prototypes optimized by Neural Ordinary Differential Equations (ODEs), and adaptive dual heads for enhanced inference. The dynamic prototypes are particularly designed to rectify prototype bias, which is a significant issue in few-shot learning scenarios. It defines the Federated Few-Shot Class-Incremental Learning (FFSCIL) problem and proposes the UOPP model to address it. UOPP uses prompt learning and dynamic prototypes to handle task interference, overfitting, and prototype bias. Experiments show UOPP outperforms existing methods on three datasets, with up to 76% higher accuracy and 67% better balance mean accuracy. It provides theoretical studies on the convergence and generalization of the UOPP method. It shows that UOPP is robust under various numbers of local clients and global rounds, with low communication costs and moderate running time.", "questions": "1. How does the scalability of the UOPP model hold up when there is a significant increase in the number of clients or data volume? Can the model adapt to the continuous addition of new categories without a significant increase in training time or resource consumption?\n\n2. How does the UOPP model perform in the face of class imbalance or distribution shift? Has the model undergone tests for generalization capabilities across different domains or tasks?\n\n3. What specific operations are used in the iterative correction process of dynamic prototypes to guide them closer to the actual data distribution? How is the number of iterations for prototype correction determined? Is there an early stopping mechanism to prevent overfitting?\n\n4. Regarding the dynamic updating of prototypes, can Neural ODEs be replaced by other optimization strategies, or how significant is the impact on the results without the ODE method, and was its computational complexity considered?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. The paper presents a novel solution for the challenging Federated Few-Shot Class-Incremental Learning (FFSCIL) problem, combining few-shot learning with federated learning. The UOPP model addresses this by integrating prompt learning, dynamic prototypes, and a dual-head classifier, showing high effectiveness.\n\n2. The UOPP model excels on benchmark datasets, surpassing existing methods and proving its real-world applicability. It's supported by a robust theoretical framework that ensures convergence and generalization.\n\n3. The UOPP model's utilization of dynamic prototype rectification through Neural Ordinary Differential Equations (ODEs) is a significant technical strength. This approach allows for the continuous optimization of prototypes, which are critical for representing new classes in few-shot learning scenarios. \n\n4. Designed with practical considerations in mind, UOPP respects data privacy, optimizes communication efficiency, and accommodates computational constraints, making it an ideal solution for federated learning environments. \n\n5. Enhancing transparency and fostering further research, the paper makes the UOPP source code publicly available.", "weaknesses": "1.The paper lacks a discussion on the future outlook and limitations of the method, which could provide a more comprehensive understanding of the research's implications, potential areas for advancement, and the constraints that may affect its practical application in various scenarios.\n\n2 In federated learning, there's always a risk of introducing bias due to non-i.i.d. data distribution across clients, which the paper might not fully address.\n\n3. There is a need for a more detailed analysis on how the model's performance is affected over extended periods and with a growing number of tasks, including the sustainability of its memory retention and its ability to adapt to new data without significant decreases in accuracy on previously learned tasks.\n\n4. Although the model aims to be efficient, the use of advanced techniques like Neural ODEs could still demand significant computational resources, potentially limiting its applicability in resource-constrained environments. The paper does not sufficiently discuss this trade-off, which could be a critical consideration for practical deployment scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "keucNx1F48", "reviewer": "Reviewer_2sKT", "summary": "The paper defines a new problem named Federated Few-Shot ClassIncremental Learning (FFSCIL) problem, where clients only hold very few samples for new classes. They develop a novel Unified Optimized Prototype Prompt (UOPP) model that utilizes task-wise prompt learning to mitigate task interference empowered by shared static-dynamic prototypes, adaptive dual heads, and weighted aggregation. The proposed UOPP significantly outperforms existing SOTA methods of FCIL, FSCIL, and CIL, on three datasets, along with deeper analysis confirms that the proposed method achieves better stability-plasticity trade-off, and robustness in different local clients and small global rounds.", "questions": "1. Please address the problems proposed in weaknesses.\n2. Table captions should be placed at the top of each table according to the ICLR template guidelines.\n3. To comprehensively evaluate the limitations in few-shot scenarios, it would be beneficial to explore additional settings between one-shot and five-shot, as well as settings with more than five shots. (optinal)\n4. Although the method claims lower resource consumption, the running time of 6.05 hours is longer than that of most other methods.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The experiments are comprehensive and solid, achieving state-of-the-art performance.\n2. The paper includes a deeper theoretical analysis, demonstrating a better stability-plasticity trade-off and robustness across different local clients and smaller global rounds.\n3. Resource analysis is thorough, covering complexity, running time, parameters, and communication costs.\n4. Clear implementation details and algorithm pseudocode are provided, along with an anonymous GitHub repository, making reproduction accessible.", "weaknesses": "1.The motivation is very weak\nThe introduction of this work is poorly written; I cannot grasp the new insights regarding the federated learning task provided by the authors. The two components of this work (1) prototype calibration and (2) introduction of prompts have already been discussed in a large number of prototype-based continual learning and prompt-based continual learning works. The authors should emphasize the uniqueness of their work.\n\n2.The lacked comparison with the recent Federated Class-Incremental Learning work\n[1] also uses Parameter-Efficient Fine-Tuning technique for Federated Class-Incremental Learning, i.e., LoRA. Thus, except for the task difference, what is the true difference between the prompt used in this paper and the LoRA used in [1]? In addition, the difference about the prototpye use between this work and [1] should also introduced.\n[1]PILoRA: Prototype Guided Incremental LoRA for Federated Class-Incremental Learning\nECCV 2024\n\n3.The comparison with prototype works in continual learning is insufficient\nPrototype Calibration is one of the important techniques in continuallearning, and there is a lot of existing work focusing on it, e.g., [2,3,4]. However, the authors do not compare their work with these efforts.\n[2] Few-Shot Class-Incremental Learning via Training-Free Prototype Calibration, ECCV 2024\n[3] Exemplar-free Continual Representation Learning via Learnable Drift Compensation, ECCV 2024\n[4] Few-Shot Class-Incremental Learning via Training-Free Prototype Calibration, ICCV 2023\n\n4.The possible data leakage issue\nAlthough the authors do not use rehearsal to correct prototypes, the support set S is needed. Essentially, samples in the supported set S are pseudo rehearsal sampled from Gaussian distributions located on prototypes, and this technique is also common in continual learning, e.g., HiDe-Prompt [5]. I am concerned that the pseudo rehearsal still violates data privacy principles.\n[5] Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CLcYtoyjh8", "reviewer": "Reviewer_2VKj", "summary": "The authors introduce Federated Few-Shot Class-Incremental Learning (FFSCIL), a new federated learning framework, along with the Unified Optimized Prototype Prompt (UOPP) model. UOPP tackles catastrophic forgetting, overfitting, and prototype bias by integrating task-specific prompt learning, unified static-dynamic prototypes, and an adaptive dual-head structure. Experiments demonstrate that UOPP surpasses state-of-the-art methods across multiple datasets, showing strong performance and robustness with low communication costs and efficient runtime.", "questions": "see weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "•\tInnovative Approach: UOPP combines prompt learning, prototype rectification, and an adaptive dual-head structure to address key challenges like catastrophic forgetting and overfitting.\n•\tComprehensive Experiments: Extensive experiments demonstrate UOPP's robustness and superior performance across multiple datasets.\n•\tOpen Source Code: The provided source code enhances reproducibility and supports further research.", "weaknesses": "•\tIn what situations is Federated Few-Shot Class-Incremental Learning (FFSCIL) actually useful, and why is it important to have a specific approach for these cases?\n•\tThe paper’s prototype rectification method might not fully solve prototype bias issues, especially when data is really limited. With few local samples, it may still struggle to perform well in non-i.i.d. settings.\n•\tThe experimental setup could be a bit limited since many of the comparison methods don’t fully match the FFSCIL scenario, which might make the performance comparisons less relevant and not show the method’s true strengths.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DuSfF23EzT", "reviewer": "Reviewer_b8Za", "summary": "The paper propose a new setting - Federated Few-Shot Class Incremental Learning (FFSCIL) where each client in the system has very few samples per class. The work focus on the prototype bias rectification process and propose a prompt-learning method named “Unified Optimized Prototype Prompt (UOPP)” which is based on static and dynamic class prototypes optimized using Neural Ordinary Differential Equations. The proposed method outperforms existing methods across three benchmark datasets and achieves better stability-plasticity trade-off.", "questions": "1. It is not clear if the ViT model used for UOPP in the experiments used pre-trained weights or not. This is not mentioned in the paper. It is important to have a fair comparison and start from the same weights for all compared methods.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper has good analysis of the prototype rectification problem with good illustrations.\n2. The paper proposed a new and interesting few-shot setting for FCIL.\n3. The method section is well-written with lot of theoretical analysis.\n4. The extensive experiments and ablation study is appreciated.", "weaknesses": "1. The writing needs to be improved and more precise with appropriate and more FSCIL references. For instance, more context and details are necessary here (line 54 - “this mechanism may violate data privacy principles since it may reveal partial information about private data of a client to another party”; line 57 - “that may breach data openness policy, where data are only open for a client at a specific moment”). The introduction section could better introduce and establish the problem statement instead of directly jumping to the solution. The motivation for the proposed FFSCIL setting, why not save exemplars and why not use synthetic samples is not clear and needs to be discussed in details.\n\n2. Existing works [1,2,3] which extensively studied prototype calibration for FSCIL which is the same concept termed as prototype rectification in this work, are completely ignored. TEEN [1] and statistics calibration [3] are very relevant single-step prototype rectification baselines which should be considered, adapted for FFSCIL and compared to the proposed method since this naive averaging method is already established for FSCIL. It is important to discuss the difference of class-similarity based rectification as in [1,3] with the proposed Neural ODE-based rectification and see its impact.\n\n3. Recent work from Federated Learning [4] used prototype-based NCM classifiers and achieved very good performance without any training. Using NCM classifier is pretty common and standard practice in FSCIL where the model is trained in the base task and the frozen model is used in all subsequent tasks to compute class means for NCM classifier. FedNCM [4] is an important baseline method missing from the discussion and comparison tables. FedNCM is the baseline which shows how much the models can perform without any further fine-tuning or learning prompts after base task.\n\n4. It is not clear what is the balance mean accuracy? Recent FSCIL papers [1,2,3] considered the harmonic mean accuracy which is quite helpful in FSCIL settings to evaluate performance of old and new classes since most of the accuracy comes from the old classes.\n\n\n[1] Wang, Qi-Wei, et al. \"Few-shot class-incremental learning via training-free prototype calibration.\" Advances in Neural Information Processing Systems 36 (2023).\n\n[2] Peng, Can, et al. \"Few-shot class-incremental learning from an open-set perspective.\" European Conference on Computer Vision, 2022.\n\n[3] Goswami, Dipam, et al. \"Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformers.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.\n\n[4] Legate, Gwen, et al. \"Guiding the last layer in federated learning with pre-trained models.\" Advances in Neural Information Processing Systems 36 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uP3cPUKVOy", "reviewer": "Reviewer_13iY", "summary": "This paper define a new Federated Few-shot Class-Incremental Learning (FFSCIL) problem which is interesting and has significante value in real model deployment. A novel Univfied Optimizaed Prototype Prompt (UOPP) method is proposed for FFSCIL problem. UOPP utilizes task-wise prompt-learning to mitigate task interference empowered by shared static-dynamic prototypes, adaptive dual heads, and weighted aggregation. The dynamic prototype tackles prototype bias by iterative rectiffcations. Comprehensive experimental results show that UOPP signiffcantly outperforms existing SOTA methods of FCIL, FSCIL, and CIL, on three benchmarks with a signiffcant gap. The proposed method achieves better stability-plasticity trade-off, and robustness in different local clients and small global rounds.", "questions": "1. Is there any error in equation (4)? As ω is a regularizer not a weight, should ωzc(s) be ω(zc(s))?\n2. Why larger round leads to worse performance for UOPP according to table 3?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. A new Federated Few-shot Class-Incremental Learning (FFSCIL) problem which is interesting and has significant value in real model deployment.\n2. An effective method UOPP is proposed for FFSCIL.\n3. Comprehensive experiments are designed and demonstrate the effectiveness in final performance, stability-plasticity balance, and robustness, which are important in class-incremental learning.\n4. The whole paper is well organized and easy to follow. The problem and idea are clearly presented.\n5. Codes are available and theoretical analysis is provided.", "weaknesses": "1. The few-shot constraint when T>0 is defined in the task level or just client level is vague. Could the authors explain whether the images assigned to each client is randomly selected from the whole training set or from a small fixed subset of the training images? If former manner, the few-shot constraint is defined in the client level and prototype rectification through different clients is able to improve the performance. If the latter manner, the few-shot constraint is defined in the task-level. Prototype rectification through globally stored representation is limited to improve the learned prototype as no more information provided.\n2. Unclear details: How Z_l is augmented to construct query sample set Q? What's the loss to train GradNet? How is ω_l determined in server-side aggregation? How is setting of D implemented in table 4?\n3. Typos: line270, \"parameter Ψ\" should be \"parameter Φ\". line362: \"10 round per class\" should be \"10 round per task\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)