Personalized Federated Learning via Feature Distribution Adaptation
Authors: Connor Mclaughlin
Lili Su
Venue: NeurIPS 2024 poster (2024)
Keywords: Federated Learning
Data Heterogeneity
Personalization
Primary area: optimization_for_deep_networks
TL;DR: Federated representation learning under a generative classifier for improved personalization under distribution shifts.

Abstract: Federated learning (FL) is a distributed learning framework that leverages commonalities between distributed client datasets to train a global model. Under heterogeneous clients, however, FL can fail to produce stable training results. Personalized federated learning (PFL) seeks to address this by learning individual models tailored to each client. One approach is to decompose model training into shared representation learning and personalized classifier training. Nonetheless, previous works struggle to navigate the bias-variance trade-off in classifier learning, relying solely on limited local datasets or introducing costly techniques to improve generalization.
In this work, we frame representation learning as a generative modeling task, where representations are trained with a classifier based on the global feature distribution. We then propose an algorithm, pFedFDA, that efficiently generates personalized models by adapting global generative classifiers to their local feature distributions. Through extensive computer vision benchmarks, we demonstrate that our method can adjust to complex distribution shifts with significant improvements over current state-of-the-art in data-scarce settings.

Personalized Federated Learning via Feature Distribution Adaptation
Connor J. McLaughlin, Lili Su Northeastern University, Boston, MA 02115 (mclaughlin.c co, w.w.....tntttttttttmmmmmmmm edu
Abstract
Federated learning (FL) distributed learning framework that leverages com- monalities between distributed client datasets to train a global model. Under heterogeneous clients, however, FL can fail produce stable training results. Per- sonalized federated learning (PFL) seeks to address this by learning individual models tailored to each client. One approach to decompose model training into shared representation learning and personalized classifier training. Nonethe- less, previous works struggle to navigate learning, relying solely on limited local datasets introducing costly techniques to improve generalization. In this work, we frame representation learning as a generative modeling task. where representations on the global feature distribution. We then propose an algorithm, pFedFDA, that efficiently generates personalized models by adapting global generative classifiers to their local feature distributions. Through extensive computer vision benchmarks, demonstrate that method adjust to complex distribution shifts with significant improvements over current state-of-the-art in data-scarce settings. Our source code is available GitHub!.
bias-variance trade-off in classifier
trained with classifier based
1 Introduction
The success of deep learning models relies heavily on access to large, diverse, and comprehensive training data. However, communication constraints, user privacy concerns, and government regula- tions on centralized data collection often pose significant challenges this requirement [31, 37, 18). To address these issues, Federated Learning (FL) [34] has gained considerable attention dis- tributed learning framework, especially for its deep networks.
properties and efficiency training
The FedAvg algorithm, introduced in the seminal work [34], remains one of the most widely adopted algorithms in FL applications [32, 45, 38, 49, 40, 7]. It utilizes parameter server to maintain a global model, trained through iterative rounds of distributed client local updates and server aggregation of client models. While effective under independent and identically distributed (i.i.d.) client data, its performance deteriorates as client datasets become more heterogeneous (non-i.i.d.). Data heterogeneity leads to the well-documented phenomenon of client drift [19], where distinct local objectives cause the model to diverge from the global optimum, resulting slow convergence [20, 28] and suboptimal local client performance [42]. Despite extensive efforts [27. 19, 46, 6] to enhance FedAvg for non-i.i.d. clients, the use of a single global model remains too restrictive for many FL applications.
Personalized federated learning (PFL) has emerged as an alternative framework that produces separate models tailored to each client. The success of personalization techniques depends on
htttps://giithuu..om//// http:/hhww...oooooooccccccchh/aa mclaughhlin/pFeddD.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

balancing the bias introduced by using global knowledge that may not generalize to individual clients, and the variance inherent in learning from limited local datasets. Popular PFL techniques include regularized local objectives [41, 26). local-global parameter interpolation [6], meta-learning [9, 16], and representation learning [35, 5, 48, 29]. While these techniques have shown significant improvements for clients under limited types of synthetic data heterogeneity (e.g., imbalanced partitioning of an otherwise i.idd. dataset), we find that current methods still struggle to navigate the bias-variance trade-off with the additional challenge of feature distribution shift and data scarcity, conditions commonly encountered in cross-device FL.
As such, we look to design a method capable of handling real-world distribution shifts, e.g., covariate shift caused by weather conditions or poor camera calibration, (see clients and 2 in Fig. 1) with limited local datasets. To this end, we approach PFL through shared representation learning guided by a global, low-variance generative classifier. Specifically, we select a probability density p with desirable statistical properties (e.g., one that admits an efficient Bayesian classifier) and iteratively estimate the global parameters of this distribution and representation layers to produce features from the estimated distribution (Fig. la).
To further navigate the bias-variance trade-off, we introduce local-global interpolation method to adapt the global estimate to the distribution of each client. At inference time, clients use their adaptive local distribution estimate in a personalized Bayesian classifier (Fig. 1b).
Contributions. We propose a novel Personalized Federated Learning method based on Feature Distribution Adaptation (DFedFDA). We contextualize our algorithm using class-conditional multivariate Gaussian model of the feature space in a variety of computer vision benchmarks. Our empirical evaluation demonstrates that our proposed method consistently improves average model accuracy in benchmarks with covariate shift or client data scarcity, obtaining over 6% in multiple settings. At the same time, our method remains competitive with current state-of-the-ard (often within 1 1%) on more general benchmarks with more moderate data heterogeneity. To summarize, our contributions are three-fold:
A novel generative modeling perspective for federated representation learning is proposed to enable new bias-variance trade-off for client classifier learning.
We propose a personalized federated learning method, PFedFDA, which leverages awareness of latent data distributions guide representation learning and client personalization.
Extensive experiments on image classification datasets with varying levels of natural data hetero- geneity and data availability demonstrate the advantages of pFedFDA in challenging settings.
2 Related Work
Federated Learning with Non-i.i.d. Data, Various studies have worked to understand and improve the ability of FL to serve heterogeneous clients. In non-i.idd scenarios, the traditional FedAvg method [34] is susceptible to client drift [19]. resulting in slow convergence and poor local client accuracy [28, 27J. To tackle this challenge, [27, 1, 21] proposed the use of regularized local objectives to reduce the bias on the global model after local training. Another approach focuses on rectifying the bias of local updates [19, 10] through techniques such as control variates. Other strategies include bassbbaann [15, 47, 31, knowledge distillation [30, 54], prototype learning [43], and contrastive learning [25]. Despite promising results non-i.i.d. data, their reliance a single global model
poses limitations for highly heterogeneous clients [17].
Personalized Federated Learning. In response to the limitations of single global model, PFL. seeks to overcome heterogeneity by learning models tailored to each client. In this framework. methods attempt strike balance between being flexible enough to fit the local distribution and relying on global knowledge to prevent over-fitting on small local datasets. Popular strategies include meta-learning an initialization for client adaptation [16, 9], multi-task learning with local model regularization [41, 26], local and global model interpolation [6], personalized model aggregation (51,50), client clustering [39, 8), and decoupled representation and classifier learning [5, 29, 35, 48, 3]. Our work focuses on this latter approach, in which the neural network is typically decomposed into the first L 1 layers used for feature extraction, and the final classification layer.
Existing works this category share feature extraction parameters between clients and rely on client classifiers for personalization. These approaches differ primarily in the acquisition of client classifiers

---

Client
Client
Cöent
Client
Bacicbone
Backbonn
Backcbone
Backbone
Pesturre
Festunes
Peaturnt
Proture
Featurns
Fearuret
Global Feature Distributions
Global Classifter
Local Feature
Personalized Classifers
Distributions
Clazs
Cluss Peature
Clars Deatunce
Class Peatures
Claze Featume
CRD Peatures
Liste Deanunce
Liate Pratunce
Backbone
Backtoone
Packbone
Backbone
Client
Clieat
Client
Client
(a) Global Representation Learning
(b) Local Distribution Adaptation
Figure I: Overview of pFedFDA. (Left) Heterogenenus elients collaboratively train representation parameters under generative classifier derived from global estimate of class fcature distributions. (Right) At test time, clients adapt the generative classifier to their feature distributions to obtain personalized classifiers.
during training, which influences representation learning. For example, FedRep [5] sequentially trains strong local classifier while holding the representation fixed, then updates the representation under the fixed classifier. FedBABU [35] proposes to use fixed dummy classifiers to align client objectives, only fine-tuning the classifier layers after the representation parameters have converged. Similarly, FedRoD [3] aims to train a generic representation model and classifier in tandem via balanced softmax loss, later obtaining personalized classifiers through fine-tuning or hypernetworks. FedPAC [48] adopts the learning algorithm of FedRep, but additionally regularizes the feature space to be similar across clients, before learning personalized combination of classifiers across clients improve generalization. However, this collaboration comes with an additional computational overhead that scales with the number of active clients. pFedGP [2] leverages shared feature extractor as kernel for client Gaussian processes. Although this approach offers improved sample efficiency, comes at the cost of increased computational complexity and reliance
inducing points set.
In a similar spirit to our method, FedEM [33] estimates the latent data distribution of clients in parallel to training classification models. FedEM estimates each client data distribution as a mixture of latent distributions, where personalized models are a weighted average mixture-specifie models. Notably, this introduces significant overhead in both communication and computation separate models are trained for each mixture. contrast, our work estimates the distribution client features in parallel training global representation model.
3 Problem Formulation
System and Objective. We consider an FL system where a parameter server coordinates with M clients to train personalized models 0,, i 1,2, ... ... Each client i has local training dataset
D; {(/] v/) where Rn and y E {1,... ,C). The model training objective in PFL is:
M
min f(A, f(01,..,0xs) AM) = M Σ F((O.).
(1)
0 P2
3

---

where Q is feasible set of model parameters, F((0,) = (a.y))> (L() is the empirical risk of dataset D. and L is a loss function of the prediction errors (e.g., cross-entropy). The client population M in FL can be large, resulting in partial client participation [17]. Let q denote the participation rate, meaning that in each round, a client participates in model training with probability q-
Following [5, 48], we approach this as problem of global representation learning and local clas- sification, which each 0g consists of shared backbone, é, responsible for extracting low-level features (= E _= = e(r)), and a local classifier, /t, for learning a client-specific mapping between features and labels. Considering this decomposition of parameters 0₂ = (h, Q), we can rewrite the original PFL objective as the following:
M min Fi(h; σ), KIEH
min GE
(2)
M
where & and H are the feasible sets of neural network and classifier parameters, respectively.
In our generative modeling framework, we consider H to be the probability simplex over (1. C}, and our algorithm uses approximations of the posterior distributions as classifiers h;. However, for fair comparison with existing work (as well as other nice properties, discussed in Section 4.1), we select a generative model of the feature space such that h; can be represented with an equivalent linear layer.
Data Heterogeneity. The data distribution of each client is a joint distribution on X' x y, which can be written as pa (z. y). Pt (=)))) or (z(z)):(g)z) to each case of data heterogeneity as follows: prior probability shift (p:(y) # pe (v)), concept drift (xly)) # py (xly)), covariate shift (pi(z) + (z)),, and concept shift (p:(ylæ) + pv (ylz)) Furthermore, the local dataset volumes D; may have quantity skew, i.e., n, + ny.
Using the terminology of [17], we refer
Methodology
In this section, we introduce PFedFDA, personalized federated learning method that utilizes a generative modeling ap- proach to guide global representation leam- ing and adapt to local client distributions. We present our method using class- conditional Gaussian model of the feature space, with additional discussion of the se- lected probability density in Section 4.1.
Algorithm 1: pFedFDA
Server initializes feature extractor de with random Gaussian weights.
Server and clients initialize feature distribution
estimates (4g) Eg)). (14 ED) as random spherical Gaussians.
for each round 0, R-1 do
Send 0y p Σ to participating clients for each active client do
Algorithm describes the workflow of PFedFDA.
0% 6%
Train of for E epochs using loss (5)
Estimate local pai, E; via (6) and (7)
Our algorithm begins with a careful initial- ization of parameters for the feature extrac- tor Φ, Gaussian means μ [ and covariance Σ (Lines 1-2). We initialize with established techniques (e.g.. [12]) such that the output features follow a Gaussian distribution with controlled variance. We similarly use a spherical Gaussian to ensure stable initialization of the corresponding generative classifier (see Section 4.1).
Estimate interpolation parameter via (9)
10
5x +- B. + (1 80m
11
X < 8 + (1 8
Send of, ".. Σ to the server end
13
14
Update of HC. My with a weighted average of each client's 's Φi 14, and E
end
At the start of each FL round r, the server broadcasts the current 60 HC Eg to each participating client. The local training of each client consists of two key components: (1) global representation learning, in which clients train à to maximize the likelihood of local features under the global feature distribution 1ry 2% (Line 7); (2) local distribution udaptation, in which clients obtain robust estimates of their local feature distribution ni; E,, using techniques for efficient low-sample Gaussian estimation (Line 8) and local-globa parameter interpolation (Lines 1)). After local training, clients send their of ". 27 to the parameter server for aggregation (Line 14).

---

In the following sections, we provide detailed explanations of each algorithmic component. In Section 4.1 we discuss the benefits of a generative modeling framework and provide the justification for our selected class-conditional Gaussian model. We outline how the resulting generative classifier can be used to guide representation learning in Section 4.2 and describe how we obtain personalized generative classifiers in Section 4.3.
4.1 Generative Model of Feature Distributions
Motivation for Generative Classifiers. central theme in FL exploiting inter-client knowledge to train more generalizable models than any client could attain using only their local dataset. This presents an important bias-variance trade-off, as incorporating global knowledge naively can introduce significant bias. Fortunately, under generative modeling approach, this bias can be naturally handled, enabling efficient inter-client collaboration.
First note that local class priors (v) can be approximated with local counts: Pr(y = c) RJ nf
Exeo nf" -;. where nf the number of local samples whose labels are c. This leaves the primary source of bias to the mismatch between local and global feature distributions Py (2(y)) and p=(2lu). Crucially, it turns out that this bias is controllable due to the dependence of on global repre- sentation parameters ø. Consequentially, we propose to minimize this bias through our classification objective, which we discuss further in Section 4.2.
Gaussian Model. In this work we approximate the distribution of latent repre- sentations using a class- caaas-soniiionall Gaussian with tied covariance, i.e., p:(2/y = c) = =Z(lp E,). We show the resulting generative classifier under this model in Eq. 4. Note that it has closed form and results in decision boundary that is linear in z. Le.. if we know the underlying local feature distribution mean and covariance, we can efficiently compute the optimal header parameters h; for the inner objective in Eq. 2.
In addition to the convenient form of the Bayes classifier, we select this distribution as the Gaussianity of latent representations is likely to hold in practice. Notably, by adopting the common technique of Gaussian weight initialization (e.g., [12]), the resulting feature space is highly Gaussian at the start of training. It has also been observed that the standard supervised training of neural networks with cross-entropy objectives results feature space that is well approximated by a condiionn Gaussian distribution [24], i.c., the corresponding generative classifier Eq. 4 has equal accuracy to the learned discriminative classifier. We provide further discussion of this modeling assumption in Appendix A.
ply = d|2)= = M(zlarE)ppy = c)
(3)
N(zle 2)p(y = =d)
log plu = cl2) x I 1 (19)
+ logply = c).
(4)
4.2 Global Representation Learning
Next, we describe our process for training the shared feature extractor áb. Similar to existing works [5, 48], our local training consists of training á via gradient descent to minimize the cross-entropy loss of predictions from fixed client classifiers. We obtain our client classifiers through Eq. 4, using global estimates of Mg Eg and local estimated priors π, For computational efficiency, we avoid inverting the covariance matrix by estimating Σ Σ-1,²
4t" with the least-squares solution w min ||Ew' - |²
The loss of client i for individual training sample (x, y) is provided in Eq. 5.
(x.vi uz E, E) Σ of (oogg(/))(2))))))))))))))))(())) E. m)
(5)
Note that for spherical Gaussian Σ = I and uniform prior we recover nearest-mean classifier under Euclidean distance. This resembles the established global prototype regularization [43], which minimizes the Fuclidean distance of features from their corresponding global class prototypes. Notably, FedPAC [48] uses this prototype loss to align client features. However, this implicitly assumes that all feature dimensions have equal variance, and additionally requires hyperparameter
5

---

À to balance the amount of regularization with the primary objective. In contrast, our generative classifier naturally aligns the distribution of client features by training e with our global generative classifier.
4.3 Local Distribution Adaptation
Local Estimation. A key component of PFedFDA is the estimation of local feature distribution param- eters, used both for model personalization and for updating the global distribution for representation learning.
Given a set of extracted features Z with n² examples per class c, a maximum likelihood esti- mate of the class means and an unbiased estimator of the covariance, respectively, are given by:
T
(6)
Σ
1 2 n
(7)
where, with slight abuse of notation, Z Raxd denotes the matrix of centered features with rows corresponding to each original feature 2j centered by their respective means, i.e.,
Ej =
(8)
EEC
Estimators Eq. 6 and Eq. may be noisy on clients with limited local data. To illustrate this, consider the common practical scenario where n4 < d. The feature covariance matrix E, client i will be degenerate; in fact, it will have a multitude of zero cigenvalues. In these cases, we can add small diagonal el Σ, and replace the non-positive-defiiit matrices with the nearest positive definite matrix with identical variance. This can be efficiently computed by clipping eigenvalues in the corresponding correlation matrix and followed by converting it back to covariance matrix with normalization to maintain the initial variance. We refer readers to [1 1] for review of low-sample covariance estimation.
Local-Global Interpolation. We introduce this fusion because even with the aforementioned correc- tion ill-defined covariances, the variance of the local estimates remains highly noisy, indicating the necessity of leveraging global knowledge. It is essential to consider that in the presence of data het- erogeneity, clients with differing local data distributions and dataset sizes have varying requirements for global knowledge.
For our Gaussian parameters μ, I, we consider the introduction of global knowledge through personalized interpolation between local and global estimates, which can be viewed as a form of prior. We provide analysis of the high-probability bound estimation error for interpolated mean estimate in simple settings in Theorem 1. The full derivation is deferred to Appendix E.
Theorem 1 (Bias- Variance Trade-Off). Let = 1. Define μ4 as the sample mean of client i's local features μ4 == 1 n zi,, and Pg the global sample mean using all N samples across M clients: n. Σj=1=
My: + EM Σ}= z!. Assume client features are independent and distributed as ~ (0;, E,), with true global feature distribution ((0g. g: Egg) We consider the use of global knowledge at client i through an interpolated estimate: Tte := Bpg + (1 B) where B E [0,1], 1). For any (0, 1), with probability at least - 8. it holds that
11 - 0:112 < (1-11)1) 3) 110, 0/112
+ 1 + 4 log 1/8 log 1/6
) (22 Tr(E,) + (1 (1-B) -82 IT(Z))
N
where c is absolute constant.
Intuitively, the estimation eиror and optimal 3 depend on the bias introduced by using global knowl- edge ||0, 0 the variance of local and global features, and the respective data volumes.
0,112- We formulate this as an optimization problem, which clients estimate interpolation coefficients 33; to combine local and global estimates of (n, Σ) with minimal k-fold validation loss:
3 E min
Σ Σ (a.v)EDa
L(E O, 8 Pr (1 88 }p 3'Sx (1 -8)E,,
(9)
05951

---

where D3 is the dataset consisting of the validation samples for the k-th fold, and (ftz. Σx) are the local distribution estimates Eq. 6 and Eq. 7 estimated using the training samples from the k-th fold. In our experiments, we avoid additional forward passes on the local dataset by preemptively storing the feature-label pairs obtained over the latest round of training.
We solve Eq. 9 using off-the-shelf quasi-newton methods (c.g., L-BFGS-B). We additionally explore using separate/ 3 terms for the means and covariance (Section 5.3) and recommend the use of single 33 term for most applications.
After obtaining B, we set our local estimates of μ4; Σ, to their interpolated versions. These estimates are then sent to the server for aggregation. Notably, the server update rule can be viewed as a moving average [52] between the previous round estimate and the client average scaled by B, reducing the influence of local noise in the global distribution estimate. At test time, clients use their local distribution estimates for inference through the classification rule in Eq. 4.
5
Experiments
5.1
Experimental Setup
We consider image classification tasks and evaluate our method on four popular datasets. The EMNIST [4] dataset is for 62-class handwriting image classification. The CIFARIO/CIFAR 100 [22] datasets are for 10 and 10-class color image classification. The Tinylm- ageNet [23] dataset is for 200-class natural image classification. For EMNIST and CIFARIO/I0 datasets, we adopt the 4-layer and 5-layer CNNs used in [48]. On the larger TinylmageNet dataset, we use the R [13] architecture. Notably, the feature dimension d for EMNIST/CIFAR CNNs is 128, and 512 for ResNet. We provide additional details in Appendix C.1.
Datasets, Tasks, and Models:
Clients and Dataset Partitions: The EMNIST dataset has inherent covariate shifts due to the individual styles of each writer. We partition the dataset by writer following [6], and train with M 1000 total clients (writers), participating with rate q 0.03. On CIFAR and Tinyet datasets, we simulate prior probability shift and quantity skew by partitioning the dataset according to Dirichlet distribution with parameters a (0.1, 0.5), where lower a indicates higher levels of heterogeneity. On these datasets, we use M = 100 clients with participation rate q = 0.3. Additional details of the partitioning strategy are provided in Appendix C.I.2.
We split each client's data partition 80-20% between training and testing.
Covariate Shift and Data Scarcity: We introduce two modifications to client partitions to simulate the challenges of real-world cross-device FI.. We first consider common sources of input noise for natural images, which may result from the qualities of the measuring devices (e.g., camera calibration, lens blur) or environmental factors (e.g., weather, lighting). To simulate this, we select ten image corruptions at five levels of severity defined in [14], and corrupt the training and testing samples of the first 50 clients in CIFAR 10/100 with unique corruption-severity pairs. We leave the remaining 50 client datasets unchanged. We refer to these datasets with natural covariate shifts as CIFARIO IITARIO-S/CIFARI 100-S and detail the specific corruptions in Appendix C.I.I.
Second, we perform uniform subsampling of client training sets, leaving them with (75%, 50%, or 25%) of their original samples. These low-sample settings are more realistic for cross-device FL. where clients rely more on knowledge sharing.
We compare pFedFDA to the following baselines: Local, in which each client trains its model in isolation; FedAvg [34] and FedAvg with fine-tuning (FedAvgFT); APFL [6]:
Baselines and Metrics:
Ditto [26]: PFedMe [41]; FedRoD [3]; FedBABU [35]; FedPAC [48]; FedRep [5]; and LG-FedAvg [29]. We report the average and standard deviation of client test accuracies.
Model Training: We train all algorithms with mini- SGD for E 5 local epochs and R 200 global rounds. We apply no data augmentation besides normalization into the range [-1, 1). For PFedFDA, use k = 2 cross-vais validatii folds to estimate single Bi term for each client. Additional training details and hyperparameters for each baseline method are provided in Appendix C.2.
5.2 Numerical Results
Performance under covariate shift and data scarcity. We first present our evaluation under natural client covariate shift with varying data scarcity in Table 1. In all experiments, pFedFDA outperforms
7

---

Table Average (standard deviation) test accuracy CIFARIO/100-S for varying proportions of training data.
Dataset
CIFARIO-S Dir(0.5)
CIFARI00-S Dir(0.5)
Samples
100
25
100
Local Only
,586(.12)
.476(116)
.461(.15)
.435(.14)
.157(.05)
.136(055)
.123(004)
.093(04))
FedAvg
.64(.13)
11)((19)
.389(.17)
.3)((11))
.233(.06)
.212(.06)
.17(.05)
.114(.0)
FodAvgFT
.82((10)
.579(.19)
.561(.17)
.526(.16
.302(.06)
.273(005)
.21((.0))
.160(.05)
APFL
.61((.1)
.520(117)
508(.16)
504(.16)
.64(.05)
.148.04)
.11((.05)
.10((004
Ditto
668(.10)
.578(118
.58(.17)
.27(.16)
.29((.55)
.27((06)
.239(.05)
.141(05)
FedBABU
602(.12)
5222.17)
495(.16)
467(.15)
117(.05)
.10(.05)
.18(,05)
107(.04)
FedPAC
.679(.99
.642(.19)
559(.16
.33(.18)
.360(07)
.30(.77)
.88(.07)
.162(005)
FedRep
612(.10)
.51(.17)
510(166
486(.16)
.176(.05)
.18(.05)
.13((00)
.100.004
FedRoD
.65(.11)
.55(.18)
537(.18)
499(.14)
.18(.55)
186(.55)
.150.04)
.115(04)
LG-FedAvg
588(.13)
.83(.16)
166(.15)
433(114)
.16(.05)
.13(.05)
.127(.05)
.09((04)
pFedMe
.679(.10)
-583(18)
.549(.1)
.523(116
.289(.06)
.268(066
.37((06)
.153(05)
PFedFDA
.724G09)
.706(.10)
.661(.11)
.595(.12)
.361(.08)
.342(.8))
.326(.08)
.227(.07
Table 2: Average (standard deviation) test accuracy on multiple datasets.
Dataset
EMNIST
CIFAR10
CIFARIO0 100
TinylmageNet
Partition
Writers
Dir(0.1)
Dir(0.5)
Dir(O.1)
Dir(0.5)
Dir(O.1)
Dir(0.5)
Local
242(.23)
865(.13)
588((11)
368(.09)
.15((05)
.27((07)
.099(.03)
FedAvg
.790(.1))
.54(.11)
.625(.07)
225(.06)
.22((05)
.155(.04)
.15(.04)
FedAvgFT
844(.10)
.00((10)
.742(.08)
.99(.09)
.14(.06)
.384(07)
.21((04)
APFL
841(.10)
882(.11)
.656(.11)
388(.09)
.19(.055
.350.09)
.177(055)
Ditto
883(.10)
898(.10)
.736(08)
504(.08)
.08(.06)
336(.07)
211(.04)
FedBABU
728(.13)
887(.11)
.67(.11)
395(.09)
.193(.0)
365(.07)
.179(.04)
FedPAC
856(.09)
.908(09)
.767(.07)
566(.08)
.38(.06)
366(.07)
1800.04)
FedRep
.735(.12)
.889(.10)
.668(.10
.39((09)
.182(.05
.39(.07)
.145(04)
FedRoD
.747(.15)
888(.11)
.713(009)
424(.08)
.224(055)
.382(.07)
.209(.05)
LG-FedAvg
.666(.13)
866(.13)
.59(.12)
.31(.09)
.162(055)
.20((07)
.105(.03)
PFedMe
842(.10)
900(.10)
.740009)
493(.08)
311(,66
.388(07)
.218(.04)
PFedFDA
844(.10)
902(.09)
.763(007)
.523(.08)
.385(07)
.384(007)
.11((04)
the other methods test accuracy, demonstrating the effectiveness of our method in adapting to heterogeneous client distributions. Additionally, PFedFDD has increasing benefit relative to other methods data-scarce settings: on CIFAR1O, we improve 4.2% over the second-best method with 100% of training samples and 6.9% with 25%. On CIFAR 100, the same improvements range from 0.1% to 6.5%. This indicates the success of our method in navigating the bias-variance trade-off.
Evaluation in more moderate scenarios. Our evaluation of all four datasets the traditional setting (no added covariate shift, full training data) is presented in Table 2. We note that: (1) our method is still competitive, always ranking within the top 3 methods, and (2) the gap between top methods is smaller than in the previous experimental setting. For example, EMNIST/CIFAR we that FedAvgFT, FedPAC, and PFedFDA are within ~ ~1% accuracy. We observe larger performance gaps for CIFAR100, with FedPAC and DFedFDA having the best results.
Results under extreme data scarcity. We present additional results at the limits of data scarcity on CIFAR CIFAR10/100 datasets in Table 3, where we assign single mini-batch (50) of training examples to each client. Notably, even as n. < d, which poses a challenge to local covariance estimation, pFedFDA clients obtain the best test accuracy, indicating the robustness of our local-global adaptation.
Generalization to new clients. We further analyze the ability of our generative classifiers to generalize on clients unseen at training time. To simulate this setting, we first train the server model model using half of the client population. We then evaluate each method on the set of clients not encountered throughout training, using their original input data, well as their dataset

---

Table 3: Results under extreme data scarcity CIFARIOCCIFARIOO Dir(O.S).
FedAvgFT
Ditto
APFL
FedBABU
pFedMe
FedRoD
FedPAC
pFedFDA
CI CIFARI00
.692(.17)
.708(17)
.698(.17)
.684(.19
.68(.16)
.631(119)
.710.16)
.725(.16)
.324.14)
14)
405(.15)
.369(11)
.30((11))
.306(.14)
.477.15)
.49((16)
Table 4: Evaluation new-client generalization on CIFARIO Dir(O.5).
New Clicats
Original
Clients
Clean
Data
Motion
Blur
Defocus
Blur
Gauss
Noise
Noise
Impulse Noise
Frost
Fog
Comp.
JPEG
Brightacss
Contrast
FodAvg
5922,07)
584,008
.912(,09
.554(08)
5688,07
575(,07)
.56KK (77) .465,00)
A47((08)
.800,07)
.557,08)
.359,10)
FedAvgFT 216),08)
(89(008)
08)
704(,09)
.695 09)
699(009
6966 (19)
6680.099
.672(.09)
.711(008)
.007(08)
688 (9)
FedBABU 7003_10)
691(.09)
685(,09) 688).08)
663(.09)
680(.99
679(.09)
.61(.10)
.661(.09)
.690(08)
.691.09)
670,099
FodPAC .277.09)
724.09)
(955.09)
.708.09)
714(.09)
7005.09) 712(.09)
(682.10)
.683(.9)
.716,009 (0)
.66(.09) .718.09)
PFedFDA .738(08)
738(.08)
.702.09)
.719,09)
729(,08)
.399,077
.725.08)
.695(,09)
.684(19))
.738,088
.733,08)
.689.09)
transformed using each corruption from CIFAR-S. Further benchmark details, including fine-tuning (personalization) procedures, are provided in Appendix C.3. As demonstrated in Table 4, our method generalizes well even on clients with covariate shifts not encountered at training time. Moreover, observe that pFedFDA has the highest accuracy on the original clients, highlighting the efficacy of structured generative classifiers when less training data is available (i.e., having 50 rather than 100 clients).
5.3 Ablation of Method Components
We conduct two studies to verify the efficacy of our local-global interpolation method. In Table 5, we see that our interpolated estimates always perform better than using only local data, indicating the benefits of harnessing global knowledge. Learning separate 3 terms for the means and covariance may be beneficial in low-sample or covariate-shift settings when the local distribution estimate may fluctuate further from the global estimate. However, using single scalar 3 appears sufficient and comes with the lowest computational cost (associated with the time to solve Eq. 9).
Table 5: Ablation study CIFARIO 100 with Dir(0.1) partition. NB denotes clients using only local data to estimate their feature distribution (3: 1). SB denotes each client estimating single B; for both the means and covariance, MB denotes clients computing 3, terms for the means and covariance separatcly. We show the average computational overhcad across all settings.
A Strategy
1) Test Accuracy
Dir(0.5) Test Accuracy
Computation Overhead
NB SB MB CIFAR100 CIFAR100 25%
CIFAR100
CIFAR100
CIFAR 25%-2% CIFAR100
(Si seconds/iter.
458((08)
382(009)
436((08)
330((06)
211((05)
.296(.06)
(0%)
.53(.08)
396(.09)
487(008)
.38((006
.266(066
.36((08)
(L 22. (2.35%)
551((08)
423(009
480008)
.379(06)
.275(06)
.373(00)
(L 36.16%)
We additionally visualize the spread of learned 3 across clients as function of their dataset corruption in Fig. 2. As expected, clients with clean datasets rely more on global knowledge (smaller 3 values) than corrupted clients. Moreover, corruptions with higher 3 values (e.g., contrast) often align with the more difficult corruptions encountered in Table 4.
5.4 Communication and Computation
The parameter count and relative communication load of our generative classifiers compared to a simple linear classifier varies depending class count C' and feature dimension d. In our experimental configurations (datasets, architectures). the overhead in total parameter count ranges from 1.1% 1.1 to 6.8%. See Appendix D.3 for additional details.
In Table 6, we compare the local training time (client-side computation) and total runtime of pFedFDA to baseline methods on CIFAR10 10. We observe slight increase in training time relative to FedAvg, which can be attributed primarily to cost of learning our parameter interpolation coefficient B. However, this increase is comparable to the existing methods and is lower than representation-

---

Learned
Motion Blur
DefocusGaussian
Shot Noise
Impuise
Frost
Fog
Brightness JPEG
Contrast
None
Blur
Noise
Noise
Client Corruption
Figure 2: Comparison client and local dataset corruption on CIFARIO-S.
learning methods FedRep and FedPAC. This demonstrates the relative efficiency our generative classifier formulation in comparison to classifiers obtained through local fine-tuning.
Table 6: Comparison of system runtime on the CIFARIO dataset.
FedAvg
APFL
Ditto
FedBABU
FedRep
FedPAC
pFedFDA
Local Training (sec)
18.42
21.41
19.97
17.41
34.82
33.60
22.58
Total Runtime (min)
61.41
71.35
66.87
58.11
116.95
146.3
77.71
Conclusion
Balancing local model flexibility and generalization remains central challenge in personalized federated learning (PFL). This paper introduces pFedFDA, a novel approach that addresses the bias- variance trade-off in client personalization through representation learning with generative classifiers. Our extensive evaluation on computer vision tasks demonstrates that PFedFDA significantly outper- forms current state-of-the-art methods in challenging settings characterized by covariate shift and data scarcity. Furthermore, our approach remains competitive in more general settings, showcasing its robustness and adaptability. The promising results underline the potential of our method to improve personalized model performance in real-world federated learning applications. Future work will focus on exploring the scalability pFedFDA and its application other domains.
Acknowledgments and Disclosure of Funding
We gratefully acknowledge the support from the National Science Foundation CAREER award under Grant No. 2340482, the Army Research Laboratory under Cooperative Agreement Number W911NF-23-2-0014 the Sony Faculty Innovation Award, and the National Defense Engineering Graduate (NDSEG) Fellowship Program. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory, the National Science Foundation, the U.S. government. The U.S. government is authorized to reproduce and distribute reprints for government purposes notwithstanding any copyright notation herein. We also thank Ming Xiang for valuable discussions and feedback on this work.
References
[1] Durmus Alp Emre Acar, Yue Zhao, Ramon Matas, Matthew Mattina, Paul Whatmough, and Venkatesh Saligrama. Federated learning based on dynamic regularization. In International Conference Learning Representations, 2021.
10

---

[2] Idan Achituve, Aviv Shamsian, Aviv Navon, Gal Chechik, and Ethan Fetaya. Personalized federated learning with gaussian processes. Advances in Neural Information Processing Systems, 34:8392-84008 8406, 2021.
[3] Hong-You You Chen and Wei-Lun Chao. On bridging generic and personalized federated learning for image classification. In International Conference on Learning Representations, 2022.
[4] Gregory Cohen, Saced Afshar, Jonathan Tapson, and André van Schaik. Emnist: extension of mnist to handwritten letters, 2017.
[5] Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting shared representations for personalized federated learning. In International Conference Machine Learning, pages 2089-2099. PMLR, 2021.
[6] Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi. Adaptive personalized federated learning, arXiv preprint 1:Xiv:2003.1346: 2020.
[7] Zhaoyang Du, Celimuge Wu, Tsutomu Yoshinaga, Kok-Lim Alvin Yau, Yusheng Ji, and Jie Li. Federated learning for vehicular internet of things: Recent advances and open issues. IEEE Open Journal of the Computer Society, 1:45-61222 2020.
[8] Moming Duan, Duo Liu, Xinyuan Ji, Yu Wu, Liang Liang, Xianzhang Chen, Yujuan Tan, and Ao Ren. Flexible clustered federated learning for client-level data distribution shift. IEEE Transactions on Parallel & Distributed Systems, 33(11):2661-2677 November 2022.
[9] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. Advances in Neural Information Processing Systems, 33:3557-3568, 2020.
[10] Liang Gao, Huazhu Fu, Li Li, Yingwen Chen, Ming Xu, and Cheng-Zhong Xu. Fedde: Federated learning with non-iid data via local drift decoupling and correction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 101 10112-10121, 2022.
[11] Yaqian Guo, Trevor Hastie, and Robert Tibshirani. Regularized linear discriminant analysis and its application microarrays. Biostatistics (Oxford, England), 8:86-100, 02 2007.
[12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision, pages 1026-1034, 2015.
[13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for im- age recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770-778, 2016.
[14] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations, 2019.
[15] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Federated visual classification with real-world data distribution. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part 16, pages 76 92. Springer, 2020.
[16] Yihan Jiang, Jakub Konečný, Keith Rush, and Sreeram Kannan. Improving federated learning personalization via model agnostic meta learning. arXiv preprint arXiv: /2488, 2019.
[17] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Ar- jun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode. Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and Trends@ in Machine Learning, 14(1-2):1-211 1-00, 2021.
[18] Georgios Kaissis, Alexander Ziller, Jonathan Hasseraa-Palmbarr Théo Ryffel, Dmitrii Usynin, Andrew Trask, Ionésio Lima, Jason Mancuso, Friederike Jungmann, Marc-Matthia Steinborn, Andreas Saleh, Marcus Makowski, Daniel Rueckert, and Rickmer Braren. End-to-end privacy preserving deep learning on multi-instinutional medical imaging. Nature Machine Intelligence, 3(6):4734444 Jun 2021.
11

---

[19] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning, In International Conference on Machine Learning, pages 5132-5143. PMLR, 2020.
[20] Ahmed Khaled, Konstantin Mishchenko, and Peter Richtárik, Tighter theory for local sgd on identical and heterogeneous data. In International Conference on Artificial Intelligence and Statistics, pages 4519 4529, PMLR, 2020.
[21] Jinkyu Kim, Geeho Kim, and Bohyung Han. Multi-level branched regularization for federated learning. In International Conference on Machine Learning, pages 11058-11073. PMLR, 2022.
[22] Alex Krizhevsky. Learning multiple layers of features from tiny images. University of Toronto, 05 2012.
[23] Ya Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015.
[24] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in Neural Information Processing Systems, 31, 2018.
[25] Qinbin Li, Bingsheng He, and Dawn Song. Model-contrastive federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10713-10722. 2021.
[26] Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and robust federated learning through personalization. In International Conference on Machine Learning, pages 6357-6368. PMLR, 2021.
[27] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429 450, 2020.
[28] Xiang Li, Kaixuan Huang, Wenhao Yang, Shusen Wang, and Zhihua Zhang. On the convergence of fedavg on non-iid data. In International Conference Learning Representations, 2020.
[29] Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B Allen, Randy P Auerbach, David Brent, Ruslan Salakhutdinov, and Louis Louis-Philippe Morency. Think locally, act globally: Federated learning with local and global representations. arXiv preprint arXiv:2001.01522 2020.
[30] Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi. Ensemble distillation for robust model fusion in federated learning. Advances in Neural Information Processing Systems, 33:2351-2363, 2020.
[31] Ximeng Liu, Lehui Xie, Yaopeng Wang, Jian Zou, Jinbo Xiong, Zuobin Ying, and Athanasios V Vasilakos. Privacy and security issues in deep learning: A survey. IEEE Access, 9:4566-4593, 2020.
[32] Nathalie Majcherezyk, Nishan Srishankar, and Carlo Pinciroli. Flow-f1: Data-driven federated learning for spatio-tempors predictions in multi-robot systems. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 8836-8842. IEEE, 2021
[33] Othmane Marfoq, Giovanni Neglia, Aurélien Bellet, Laetitia Kameni, and Richard Vidal. Federated multi-task learning under mixture of distributions. Advances in Neural Information Processing Systems, 34:1 14:15434-15447, 2021.
[34] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas Communication-eficiie learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273-1282. PMLR, 2017.
[35] Jaehoon Oh, SangMook Kim, and Se- Young Yun. FedBABU: Toward enhanced representation for federated image classification. In International Conference on Learning Representations, 2022.
12

---

[36] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32, 2019.
[37] Kilian Pfeiffer, Martin Rapp, Ramin Khalili, and Jörg Henkel. Federated learning for computa- tionally constrained heterogeneous devices: A survey. ACM Computing Surveys, 15(14))11-27, July 2023.
[38] Swaroop Ramaswamy, Rajiv Mathews, Kanishka Rao, and Françoise Beaufays. Federated learning for emoji prediction a mobile keyboard. arXiv preprint arXiv: urXiv:I906..00 1906.04399,, 2019.
[39] Felix Sattler, Klaus-Robert Müller, and Wojeiech Samek. Clustered federated learning: Model- agnostic distributed multitask optimization under privacy constraints. IEEE Transactions on Neural Networks and Learning Systems, 32(8):3710-3722 2020.
[40] Micah Sheller, G Anthony Reina, Brandon Edwards, Jason Martin, and Spyridon Bakas. Multi-institutional deep learning modeling without sharing patient data: A feasibility study on brain tumor segmentation. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16. 2018, Revised Selected Papers, Part I 4, pages 92-104. Springer, 2019.
[41] Canh T Dinh, Nguyen Tran, and Josh Nguyen. Personalized federated learning with moreau envelopes. Advances in Neural Information Processing Systems, 33:21394-21 21405, 2020.
[42] Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems, 2022.
[43] Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. Fedproto: Federated prototype learning across heterogeneous clients. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8432-8442 8440, 2022.
[44] Roman Vershynin. High-ddimensionl probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018
[45] Chunnan Wang, Xiang Chen, Junzhe Wang, and Hongzhi Wang. Atpfl: Automatic trajectory prediction model design under federated learning framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3563-6572, June 2022.
[46] Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in Neural Information Processing Systems, 33:761 1-7623, 2020.
[47] Lixu Wang, Shichao Xu, Xiao Wang, and Qi Zhu. Addressing class imbalance in federated learning. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10165-10173, 2021.
[48] Jian Xu, Xinyi Tong, and Shao-Lun Huang. Personalized federated learning with feature alignment and classifier collaboration. In The Eleventh International Conference on Learning Representations, 2023.
[49] Timothy Yang, Galen Andrew, Hubert Fichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ramage, and Françoise Beaufays. Applied federated learning: Improving google keyboard query suggestions. arXiv preprint
2018
[50] Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Fedala: Adaptive local aggregation for personalized federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1 11237-11244 2023.
[51] Michael Zhang, Karan Sapra, Sanja Fidler, Serena Yeung, and Jose M. Alvarez. Personalized federated learning with first order model optimization. In International Conference on Learning Representations, 2021.
13

---

[52] Sixin Zhang, Anna Choromanska, and Yann LeCun. Deep learning with elastic averaging sgd, 2015.
[53] Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Learning to generate novel domains for domain generalization. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVI 16, pages 561-578. Springer, 2020.
[54] Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for het- erogeneous federated learning. In International Conference on Machine Learning, pages 2288--12889 PMLR, 2021.
14

---

A Limitations
The selected class-conditional Gaussian distribution may not work well for all neural network architectures. For example, if the output features are the result of an activation such as ReLU, truncated Gaussian distribution may be better model. Future work can look to exploit knowledge of the neural network architecture to improve the accuracy of the feature distribution estimate. this work, we leverage the insights from a fusion global and local feature space. As in many applications there is often underlying cluster structure between clients datasets, future works may explore the identification and efficient estimation feature distributions client clusters, in order to reduce the degree of bias introduced in client collaboration.
B Broader Impacts
Federated learning has become the main trend for distributed learning in recent years and has deployed in many popular consumer devices such as Apple's Siri, Google's GBoard, and Amazon's Alexa. Our paper addresses the practical limitations of personalization methods in adapting clients with covariate shifts and/or limited local data, which is a central issue in cross-device FL applications. We unaware of any potential negative social impacts of our work.
C Details of Experimental Setup
All experiments are implemented in Porch 2.1 [36] and were each trained with a single NVIDIA A100 GPU, Compute time per experiment ranges from approximately 2 hours for CIFARIO/10 and 20 hours for TinylmageNet. Code for re-implementing our method is provided the following GitHub URL: https://ttti /gtittittt../...//// com/c mmc.aaaggii///eeeFF..
C.1 Dataset Description
The EMNIST [4] dataset contains over 730,000 28 28 grayscale images of 62 classes of handwritten characters. The
100 [22] datasets contain 60,000 32x32 color images in 10 and 100 different classes natural images, respectively. TinyImageNet [23] contains 120,000 64 x 64 color images of natural images.
C.1.1 CIFAR-S Generation.
We implement the following 10 common image corruptions at 5 levels of severity as described in [14]: Gaussian noise, shot (Poisson) noise, impulse noise, defocus blur, motion blur, fog, brightness, contrast, frost, JPEG compression. We apply a unique corruption-severig pair to all samples of the first 50 clients.
C.1.2 Non-i..d. Partitioning.
On CIFAR and TinyImageNet datasets, we distribute the proportion of samples of class C across M clients according to a Dirichlet distribution: m Dir (a), where we consider a E (0.1, 0.5) as in [30].
We provide a visualization of Dirichlet partitioning strategies on CIFARIO below. The size of each point represents the number of allocated samples. Notably, increases, Dir(a) becomes less heterogeneous.
C.2 Training Settings
All methods are trained using mini-batch SGD for 200 global rounds with local epochs of training.
We use a fixed learning rate of 0.01, momentum of 0.5, and weight decay of 5e-4. The batch size is set to 50 for experiments, except for EMNIST, where we use batch size of 16. We sample set of active clients uniformly with probability 4=.33 for CIFAR and TinylmageNet and q=0.03 for EMNIST. The global round of training employs full client participation. We split the data of each client 80-20% between training and testing.
15

---

(a) CIFARIO Dirichle((O.1)
(b) CIFARIO Dirichlet(O.5)
Figure 3: Comparison of Dirichlet Partitions on CIFARIO.
Hyper-parameters, For APFL, we tune over [0.25, 0.5, 0.75, 1.0], and set 0.25. For PFedMe, we tune À over [1.0, 5.0, 10.0, 15.0] and set 5.0. For Ditto, use five local epochs for personalization and tune μ over [0.05, 0.1, 0.5, 1.0, 2.0] and set 1.0. For FedRep and FedBABU, we use five local epochs for training the head parameters. For FedPAC, we tune over [0.1, 0.5, 1.0, 5.0, 10.0), and set À = 1.0. FedPAC uses one local epoch for training head parameters with higher learning rate of 0.1, following the original implementation.
C.3 Evaluation on New Clients
Our fine-tuning procedure on new clients largely follows the methodology above. For FedAvgFT, we fine-tune the global model for five local epochs. For FedBABU and FedPAC, personalize model in 2 different ways and report the best result: (1) fine-tuning only the head for local epochs, and (2) fine-tuning both the body and head for 5 local epochs. For pFedFDA, each new client estimates their local interpolated statistics (i.e., lines 8-1 611 Algorithm I) to obtain personalized generative classifier.
For our covariate shift evaluation, we apply a medium severity corruption (level 3) all samples.
D Additional Results
D.1 Multi-Domain FL
In Table 7, we present results on the DIGIT-5 domain generalization benchmark [53]. This presents an alternate form of covariate shift, the data from each client is drawn from one of 5 datasets (SVHN, USPS, SynthDigits, MNIST-M, and MNIST). In particular, we use 20 clients trained with full participation, and assign 4 clients to each domain. Within each domain, we use the Dirichlet(O.5) partitioning strategy to assign data to each client. We observe that pFedFDA is effective in all settings, but has the most significant benefits over prior work in the low-data regime.
Table 7: Results on multi-domain DIGIT-5 benchmark for varying data volumes.
DIGIT-5 Samples
25
Avg. Improvement
Local
76.84
86.97
88.51
FedAvg
81.75 (+4.91)
85.09 (+1.98)
87.41 (+0.44)
88.19 (+0.32)
1.91
FedAvgFT
85.61 (+8.77)
88.72 (+5.61)
90.75 (+3.78)
91.73 (+3.22)
5.34
Ditto
83.85 (+7.01)
85.53 (+2.42)
87.43 (+0.46)
88.80 (+0.29)
2.54
FedPAC
82.78 (+5.94)
87.94 (+4.83)
91.12 (+4.15)
91.04 (+2.53)
4.36
PFedFDA
86.54 (+9.70)
90.05 (+6.94)
90.75 (43.78)
91.56 (+3.05)
5.86
D.2 Effect of Local Epochs
In many FL settings, we would like clients to perform more local training between rounds to reduce communication costs. However, too much local training can cause the model to diverge. In Fig. 4, we compare the effect of the local amount of epochs for CIFAR100 and 10F-R100-S-2544 sample datascts. We observe that (1) PFedFDA outperforms FedAvgFT at all equivalent budgets of E, (2)
16

---

both methods follow exhibit general plateau in accuracy after E = 5, and (3) pFedFDA learns much faster than FedAvgFT, with significantly higher accuracy for 1.
40
35
Test Test Accuracy (9)
30
25
20
15
FedAugFT
10
pFedFDA
CIFAR100 Dir(0.5)
CIFAR100-5 Di)(0.5)
10
20
Local Epochs
Figure 4: Comparison of average test accuracy with varying local epochs on CIFAR 100.
D.3 Communication Load Examples
In Table 8, we compare the number of distinct parameters in our Gaussian estimates to that of typical linear classifier for the models and datasets used in this paper, along with some additional examples. We display the resulting overhead relative to the base parameter count of the shared representation backbone.
Table 8: Comparison of communication load (paramctors/tter) between our Gaussian distribution parameters (j,, Σ) and standard linear classificrs.
Linear Classifier (C x (d 1))
Gaussian (Ja, E)
Parameters
Backbone
(C x d (d + d))
A Overhead
EMNIST-CNN (EMNIST, C = 62)
115776
7998
16192
6.620%
CIFAR-CNN
106400
12900
21056
6.837%
(CIFAR100, C 100)
ResNet18
11167212
102600
233728
1.164%
(TinylmageNet. C' 200)
MobileNerV3-Smman (ImageNet, C = 1000)
927008
1615848
1548800
-2.637%
D.4 Runtime of Method Components
In Table 9, we evaluate the proportion of each local iteration of pFedFDA associated with each line of algorithm. Network Passes refers to the time taken to train the base network parameters á (Line 7 of Alg. 1). Mean/Covariance Est. refers to the time taken to estimate the local mean and covariance from features extracted during model training (Line 8 of Alg. 1). Interpolation Optimization refers to the time taken to optimize the local coefficient 3 (Line of Alg. 1). Overall, we find that the majority of the overhead of our method comes from estimating the interpolation parameter B.
Table Percentage (average (std)) of the local training time associated each component of our algorithm.
Network Passes
Mean/Covariance Est.
Interpolation Optimization
CIFARIO CIFAR100
84.88 (6. 6.799)
0.765 (0.281)%
14.36 (6.62)%
77.70 (5.75)%
2,861 (0.899)%
19.43 (5.70)
Tiny ImageNet
87.41 (1.50)%
2.701 (0.659)%
9,886 (1.14)%
17

---

E On the Bias-Vare Variance Tradeoff
This section justifies the bias- tradeoff under some simplified technical assumptions. For simplicity, we assume that at any given round, the extracted feature vectors for a class are independent. We illustrate the bias-variance tradeoff in estimating the mean feature of given class e round t.
Proof of Theorem 1. For ease of exposition, we drop the time index and class index.
Let be arbitrary client with local dataset size nt, of class Let N be the total data volume of class e over the entire FL system. Assuming that the distribution of client i's features z follow mul- tivariate Gaussian distribution N 22,, and the global feature distribution follows (0, 2) where 8g E (iE(A( n4), Eg Ex= M parameters.
2/E,/( CiE n4)2. Note 0g 0g are deterministic
We denote the local and global mean estimates
Ths
1
M
n
P4
and Pg
T4
N
Let Pi be the local estimate that interpolates between local and global knowledge, defined as 3p: + (1 g)
(10)
We will focus bounding the high probability local estimation error -| E [me]ll2- Note that (10) can be further expanded
74 M4
n2
e4 = Bpg + (1 (1-3) 3)
+
My
N
(B (1 3) T4i
M4 (1 3)
Σ
H4
1
= 7 + (1 - TL
where y
3 3 (1 (11-8) N(0,, E.) and j ~ N No, n.b.
3) n and ja = N Σ+ ΣØ nr 1
Thus μ4 have
NE, R. N
Since μi and j are independent, we
N
- 0, N - 0.),
- + 3)2
NE, n.E.
14
N2
Let g ~ (O, I). We have
To - 0, (1 (1 - 0:) + 53122
where 11/2 is the square root matrix of E == 222 n. E, + (1 - B)2 NE, It holds that
N²
l| 0:113 = 13)2 0:112 + 2(1 /3) (0g 10,-0,551133
(51
Taking the expectation with respect to the randomness in the Gaussian random variable g and by the law of total expectation, we have
E 21 B) (0, 0,, )3120) 2(1 3) (0s 0,, 21/2B [o]) 0,
18

---

and
E ( 11/2 g,
g))
@ Eg
+ E g T (1 - 3)² NE,, - -n;Z
9 0' 14
Eg
N²
N Tr(Eg) ni Tr(E;)
Tr(E,) + (1 - 3)2
N
where equality (a) holds because By Hanson- WWrigt inequality [44, Theorem 6.2], we conclude that with probability at least 1 8 (for any given E (0, 1)),
= E as 1//2 is symmetric.
1 - 0:112 1 (1 8)² |10g 0:112
82 +
1 2 B) Tr(2;)) + (1- B)² / Tr(2ig)
N
B2
23(1 3)
Tr(,)) + (1 3)² 1
log 1/0 log 1/6
14 +
Tr(Egg)
max
N
c
(
S (1 B)² 110g - 0.112
B + 21(1 - B) Tr(E,) + (1 - -)
Tr(Eg)
n
N
82
Tr(,,) + (1 3)2 1
log 1/8
log 1/8
+ 2
Tr(En)
)
max
144
N
F
c
(
$
(1 - 8) 10, 0.11
23 - 82
Tr(E,) + (1- Tr(Eg)
14
N
log 1/8
log 1/8
(28 - 82
VIHEE + (1- 3)2
4
+
Tr(27))
c
N
(d)
(1
110g -))3
0bl1
log 1/0 )] 2/3 Tr(2;) + (1 (1-3)² Tr(2ig)
1 + 4 log 1/8
n
N
where >> is some absolute constant, inequality (b) holds as a-b < (2tb)2 and N 2 n., inequality
(c) holds because of triangular inequality ||A + BllF S || + || that || || VTATT
Ir (AT A) V TT(A2) if matrix A is symmetric, and that max(a, b} < + 6, inequality (d) holds because Tr(A?) < (Tr(A))² for positive semidefinite matrix A and that Tr(E,), 8 2 0, and Tr(E.)) are by definition non-negative.
The first term (1 (1-3) 2) 100 11 is the bias introduced when client uses global knowledge; the smaller the 8, the more bias introduced. The last term reveals the interaction of 3 and the tradeoff between local and global variance. When 8 approaches O, we have the global feature variance Tr(E) reduced by the average of N global samples. When B approaches I, we have local feature variance Tr(E,) reduced by the average of only ni local data. Thus the bias-variance tradeoff on client crucially depends on the degree local-global distribution shift, 1 0, 1 the local data volume n, and its quality (i.e., E,), and the volume and quality of the data across clients N, Eg.
19

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
The scope of the paper is on an important topic of client model personaliza-
tion in federated learning. We faithfully state our contributions both the abstract and introduction.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitations and considerations of our Gaussian modelling of the feature space both in the main text and with additional notes the appendix. While the focus of the paper in improving client personalization in the challenging setting of data scarcity and client distribution shift, we additionally benchmark our method in more general settings to demonstrate the widespread applicability of our work. Finally, provide an assessment of the communication and computation overhead of our method compared to state-of-art approaches.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We provide the key assumptions in the main text. The missing proof is deferred Appendix E.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide detailed experimental setups and review the hyperparameters in Section 5,4 and Appendix C.2. We additionally provide code and instructions to train our method.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: Our evaluations are based on open-accessed datasets that are publically avail- able. An official implementation code is provided in the supplementary materials.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide detailed experimental setups and review the hyperparameters in Section 5.4 and Appendix C.2. We additionally provide code and instructions train our method.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
20

---

Answer: [Yes]
Justification: For our main experiments we include the standard deviation of client accuracies, and include std error bars in our ablation visualization of the method components.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please find the software/hardware specifications in Appendix C.
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: /tt///uuurrrrrre htcc/public/Bticsauuiiiiiiieiiie
Answer: [Yes]
Justification: The NeurlPS code of ethics is strictly enforced throughout our research.
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have discussed the broader impacts of our work in Appendix B. Please find details therein.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The existing assets used in this paper has been adequately cited credited to.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: Our attached code well documented and comes with a README file indicating how reviewers may set up our experiments and train the proposed method.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
21

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects
22


Reviews:
[{"review_id": "BEXtWkHyY9", "reviewer": "Reviewer_sWFJ", "summary": "This paper introduces pFedFDA, a personalized Federated Learning (pFL) method designed to address the issue of client heterogeneity in federated learning. pFedFDA combines global knowledge through server aggregation with local knowledge through client-specific training and distribution estimation, enhancing the model's personalized performance on each client. The authors propose an algorithm that efficiently generates personalized models, demonstrating significant improvements in data-scarce settings through extensive computer vision benchmarks. The paper is well-structured, and the writing is clear, making it easy to follow the authors' arguments and experimental results.", "questions": "The main concerns and questions are listed in the weaknesses. Please provide answers to address these concerns.", "limitations": "The authors adequately addressed the limitations.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The method of decomposing model training into shared representation learning and personalized classifier training, followed by adaptation to local feature distributions, is both innovative and promising for handling non-i.i.d. data in FL environments.\n2.The paper provides strong empirical evidence through comprehensive experiments on various datasets, showcasing the superiority of pFedFDA in challenging distribution shift and data scarcity scenarios.\n3.The paper is well-written, with a clear structure that logically progresses from the introduction of the problem to the presentation of the methodology and results.", "weaknesses": "1. In this paper, the mean value and covariance are utilized as the characteristic distribution of the data. Have you explored other statistical measures to describe this distribution?\n2. In the ablation study, the results obtained by calculating multiple β values were unexpectedly lower compared to using a single β value. Intuitively, multiple β values should provide a more comprehensive understanding of the data distribution and thus yield better results. However, the experimental findings do not support this expectation, and there is no clear explanation for this discrepancy.\n3.Note that pFedFDA uses the same feature extractor for all clients but employs heterogeneous classifiers. A key feature of pFL is the heterogeneity of client models. pFedFDA is somewhat limited in this regard. Are there methods to handle heterogeneous feature extractors in pFL scenarios?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1 (Choice of Statistical Measures)** We selected the mean and covariance as the class-conditional Gaussian model is uniquely defined by these parameters. While it would have been possible to communicate estimates of the inverse-covariance matrix, this adds unnecessary computation which we avoid in our implementation (refer to line 197). \n\nIf a distribution other than the class-conditional Gaussian is considered for modeling $p(z|y)$, it may be reasonable to send different sufficient statistics or estimate higher moments. \n\n**W2 (Discussion of Beta Value Ablation)** Thanks for the interesting question. Indeed, by using separate coefficients for the means and covariance, the interpolated classifier is more flexible towards fitting the local distribution. While we optimize beta using k-fold validation, this is done over the set of training features, so there is still potential for over-fitting. We chose to not optimize over a separate validation set to make our results comparable to existing approaches which do not require additional held-out data. \n\n**W3 (Limitations of pFedFDA Personalization)** We would first like to point out that using a shared backbone is not necessarily a limitation, but a common feature of recent representation-learning methods for PFL (e.g., FedRep [1], FedBABU [2], FedPAC [3]). This parameter-sharing approach learns generalizable features and simplifies the personalization task of each client to the final classification layer. \n\nStill, if model heterogeneity is desired (e.g., due to different device computation resources), pFedFDA could be extended to these settings by adopting an approach similar to FedProto[4]. Specifically, clients could collaborate on the means and covariance of the global feature distribution, without broadcasting or aggregating the backbone model. As discussed in Sec. 4.2, our generative classifier objective is similar to the regularization term and inference objective of FedProto, where we use an estimated covariance matrix rather than implicitly assuming the covariance to be the identity matrix.\n\n[1] Exploiting Shared Representations for Personalized Federated Learning (Collins et al., ICML 2021)\n\n[2] FedBABU: Towards Enhanced Representation for Federated Image Classification (Oh et al., ICLR 2022)\n\n[3] Personalized Federated Learning with Feature Alignment and Classifier Collaboration (Xu et al., ICLR 2023)\n\n[4] FedProto: Federated Prototype Learning across Heterogeneous Clients (Tan et al., AAAI 2022)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "42uETsSBuQ", "reviewer": "Reviewer_BoVr", "summary": "This paper uses Class-Conditional Gaussian Model to formulate the latent representation of the global generative part; for the other part,  a personalized federated learning algorithm pFedFDA is designed via Federated Distribution Adaptation. This paper then proves a bound on the bias-variance trade-off of pFedFDA under the assumption of independent and Gaussian distributed local dataset features.  And the performance of pFedFDA against dataset scarcity and heterogeneity is presented.", "questions": "1.\tCan the Class-Conditional Gaussian Kernel represent the optimal Gaussian distribution?\n2.\tFurther explain the experiment Tab. 1. How can this relate to the advantage of pFedFDA on bias-variance balancing? Also, other models (i.e. APFL, Ditto) have a stable performance enduring the dataset scarcity. How can this be explained?\n3.\tIn Tab. 2, the line of FedRep, please double check the value of the last cell .145(0.4). Is that correct?\n4.\tDo you perform normality test on the training set of agents? If not, can you give a further explanation of the feature distribution of datasets to match the assumption of Theorem 1?", "limitations": "1.\tThe assumption of Theorem 1 is strong. It would be better to consider the skewed distribution, and even long-tailed distributions. Based on this analysis, the result in Tab. 3 could be expanded to more general few-shot or one-shot learning cases, with a modified version of pFedFDA. \n2.\tThe design of pFedFDA is still not robust to heterogeneity and is not capable of better personalization.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.\tThe introduction and related work are well-organized. The explanation on the replacement of generative global model of the embedding module is convincing. \n2.\tThis paper gives a solid proof on the analysis of bias-variance trade-off.", "weaknesses": "1.\tThe experiment is concentrated on the CIFAR-10 and CIFAR-100. However, it is also the dataset where pFedFDA has best performance on (Tab. 2). Also, the robustness of pFedFDA against strong heterogeneity is not good. The weakness of pFedFDA against heterogeneity is still a limitation in this paper. \n2.\tThe assumption of Theorem 1 is very strong, however, this paper does not give an evaluation on the datasets in the experiment of longtailed-ness or normality. Also, the basic result of Theorem 1 is about the upper-bound of local bias, there is no matric in the experiment to compare the performance of bias-variance balance among the algorithms, especially in the scarcity experiment (Tab. 1, 3). \n3.\tThis paper does not explain the connection between bias-variance balance and robustness against data scarcity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1 (Robustness to Data Heterogeneity)** We note that while Tab. 1 and Tab. 3 are based on CIFAR datasets, these evaluations introduce the additional challenges of client covariate shift (via natural image corruptions) and data scarcity. Our strong performance in these settings indicates that pFedFDA is robust to realistic sources of covariate shift, in addition to quantity skew and prior probability shift introduced via Dirichlet-based data partitioning. \n\nFor more discussion of Tab. 2, please refer to our response W1 to reviewer WK31.\n\n**W2 (Gaussian Assumption)** The application of the multivariate central limit theorem to feature representations is a common step in the analysis of neural networks, e.g., [1,2] study their relationship with Gaussian Processes. Moreover, it has been observed that the distribution of features from trained neural networks is well approximated by a class-conditional Gaussian, with a Gaussian discriminant analysis classifier having comparable accuracy to the softmax classifier used for training [3]. This has led to the widespread usage of class-conditional Gaussian feature space approximations in the literature on out-of-distribution detection [3, 4, 5].\n\n[1] Deep Neural Networks as Gaussian Processes (Lee et al., ICLR 2018)\n\n[2] Dropout as a Bayesian Approximation (Yal et al., ICML 2016)\n\n[3] A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks (Lee et al., NeurIPS 2018)\n\n[4] A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection (Ren et al., NeurIPS 2019)\n\n[5] Exploring the Limits of Out-of-Distribution Detection (Fort et al., NeurIPS 2021)\n\n**W2/W3 (Connection of Bias-Variance Tradeoff and Data Scarcity)** Thank you very much for pointing this out. We will add further explanations on their connection in the camera-ready version if accepted. Intuitively, limited client data can lead to high variance and poor generalization in local models, encouraging collaboration with other clients for lower variance. However, collaborative estimates (e.g., FedAvg or local-global interpolation) introduce potential bias when client data is non-iid. Theorem 1 theoretically captures this intuition on the bias-variance tradeoff of interpolated estimates and their effect on generalization error, which is implicitly measured by client generalization accuracy in our experiments.\n\n**Q1 (Questions on the Class-Conditional Gaussian Distribution)** Please refer to our response to W2 on the justification of the class-conditional Gaussian model. \n\n**Q2 (Setup of Tab. 1)** In Tab. 1, we introduce real-world covariate shift between client distributions by corrupting the inputs of the first 50 clients with natural image alterations [6]. This simulates FL settings where clients collect data from different devices and environments, introducing input noise not present in curated benchmark datasets. For details on image corruption, see Appendix C.1.1. Comparing Tab. 1 and 2, we see this covariate shift significantly reduces the generalization performance of FL methods.\n\nSince half of the clients' data no longer matches the CIFAR10 distribution, there's increased bias in using global knowledge (as discussed in W2). \n\nWe assess methods under varying data scarcity to evaluate their ability to navigate the bias-variance tradeoff discussed in W2.\n\nRegarding stability in low-sample settings, our method shows no consistent variance advantage/disadvantage over APFL and Ditto, but has significantly higher average client accuracy.\n\n[6] Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations (Hendrycks et al., ICLR 2019)\n\n**Q3 (Correction to Tab. 2)** We thank the reviewer for their attention to detail. This was a typo and the entry .145(0.4) should instead read .145(.04).\n\n**Q4 (Gaussian Assumption in Practice)** As discussed in W1, we use the class-conditional Gaussian assumption, common in prior literature for describing latent representation distributions in practical settings. The empirical success of our generative classifiers supports its applicability in a variety of FL scenarios. Based on these findings, we conjecture that normality testing is not a prerequisite for applying our method.\n\nHowever, to provide a reference measure of Gaussianity in our experiments, we train FedAvg on CIFAR10 Dir(0.5) and perform the Henze-Zirkler normality test on client features centered by class means, and observe that 89/100 clients follow a class-conditional Gaussian distribution at significance level p=0.05. \n\n[7] A class of invariant consistent tests for multivariate normality (Henze and Zirkler, Communications in Statistics-Theory and Methods, 1990)\n\n**L1 (Theorem 1 and Alternative Distributions)** For an explanation of the class-conditional Gaussian model and assumptions in Theorem 1, please refer to our response W2/W3.\n\nWe recognize that other models may perform better in some settings (as discussed in Appendix A), but we defer this optimization to future work. Based on our empirical results, the class-conditional Gaussian model appears reasonable for practical use.\n\n**L2 (Robustness to Heterogeneity and Personalization Performance)** We apologize for any confusion and will make an effort to make the robustness of our method to data heterogeneity more clear. \n\nIn this work, we evaluated our method in the presence of various types of data heterogeneity, including quantity skew and prior probability shift (via Dirichlet-partitioning and sub-sampling), as well as covariate shift (via natural image corruptions [6]). Our empirical results indicate the ability of pFedFDA to generate personalized client models that are robust to heterogeneity, notably in the more challenging settings of data scarcity and client covariate shift.\n\nOur rebuttal includes an additional experiment (Tab. 1) which shows that pFedFDA also generalizes well to new clients, even for clients with covariate shifts not seen at training time.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "O7J7J7QSXx", "reviewer": "Reviewer_3xQF", "summary": "The paper introduces a personalized Federated Learning (FL) method that adapts global generative classifiers to local feature distributions. The authors show that their method can handle complex distribution shifts for computer vision tasks.", "questions": "- In Fig. 1, the colors representing client 1 and client 2 are not easily distinguishable.\n- In Eq. (4), wouldn’t the cost of the covariance matrix inversion be high? Could the server calculate it once and send it to the client to reduce the client’s computational cost, even if it increases communication cost?\n- In Eq. (9), should “\\in min” be replaced with “= \\argmax”?\n- If the client’s generative classifier’s mean and variance are sent to the server, wouldn’t the privacy risk increase compared to sending only the weight of the conventional linear classifier?\n- If the generative classifier is not used and only the weight parameter from the conventional linear classifier is interpolated locally, could the performance be expected to be mid-range between FedAvgFT and pFedFDA?\n- In experiments like Table 2, where there is less covariate shift between clients within the federation, FedPAC performs better. Could the degree of covariate shift within the federation be determined by looking at the difference in the generative classifier’s statistics coming from the client, and then choose the appropriate FL method?", "limitations": "Please refer to the weaknesses section.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper proposes a personalized FL method that uses a generative classifier by considering bias-variance trade-off.\n- The authors conduct extensive experiments.\n- The paper is well-structured and easy to understand.", "weaknesses": "- Sharing the statistics of the generative classifier could increase privacy risks.\n- The algorithm’s additional computational cost (in Algo.1 L8-9) could increase with larger local data. Calculating the local covariance matrix for larger data and the covariance inversion matrix for each beta value could be computationally intensive.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1/Q4 (Privacy Concerns of Sharing Gaussian Sufficient Statistics)** We appreciate the reviewer's comments and think it is important to discuss the privacy implications in our work. \n\nThe transmission of client feature statistics to the parameter server does not immediately raise high privacy risks as these statistics are not calculated directly from client raw data but from a mapping in a latent space embedded by a neural network. In addition, clients broadcast only the interpolated estimates of the mean and variance, and the local interpolation parameter $\\beta$ is not shared with the server. Thus, a potential attacker does not have direct access to the true local statistics. \n\nOn the other hand, it is difficult to make any guarantees without using explicit privacy protection techniques, such as homomorphic encryption [1] or differential privacy (DP) [2]. Both of these techniques are compatible with our method. Notably, our results in the low-sample regime (Tab. 3) indicate that pFedFDA can tolerate noisy distribution estimates, which is promising for future integration with recent techniques [3,4,5] for efficient DP estimates of the mean and covariance.\n\n[1] Privacy-Preserving Deep Learning via Additively Homomorphic Encryption (Phong et al., IEEE Transactions on Information Forensics and Security 2018)\n\n[2] Personalized Federated Learning With Differential Privacy (Hu et al., IEEE Internet of Things Journal 2020)\n\n[3] Mean Estimation with User-level Privacy under Data\nHeterogeneity (Cummings et al., NeurIPS 2022)\n\n[4] Differentially Private Covariance Estimation (Amin et al., NeurIPS 2019)\n\n[5] Differentially Private Covariance Revisited (Dong et al., NeurIPS 2022)\n\n**W2/Q2 (Computational Cost Associated with Covariance Estimation)** The added cost of client covariance estimation is reasonable in comparison to the base computation of training a neural network. Additionally, we avoid the more expensive calculation of the inverse covariance by solving a least-squares problem instead (refer to L197). This has a reduced complexity of O($cd^2$) compared to O($d^3$) for a naive matrix inversion algorithm. In our local training, the local covariance matrix is estimated once, and the optimization of the interpolation parameter re-uses this estimate and only has to recompute the least-squares problem for each value of beta.\n\nIn Tab. 2 of the attached rebuttal document, we measure the percentage of local training time associated with the base network training (forward/backward passes), the estimation of the local mean and covariance, as well as the optimization of the interpolation coefficient. We note that estimating the mean and covariance is less than 3\\% of the total runtime of each round. \n\nFrom this observation, we run experiments in which the interpolation parameter is only estimated every $T$ rounds, as this is the primary overhead introduced in pFedFDA. While updating the interpolation coefficient every round results in the best performance, we observe a similar accuracy using $T=2$ or $T=3$, which would result in a substantial reduction of the pFedFDA overhead. Detailed results for this study can be found in Tab. 3 of the attached rebuttal document.\n\n**Q1/Q3 (Figure 1 and Equation 9 Presentation)** We appreciate the feedback and will improve upon the clarity of our work by adjusting the client color palette and making the notation of Eq. (9) consistent with the rest of the paper.\n\n**Q5 (Generative vs. Discriminative Classifier Interpolation)** Thanks for the interesting question. We agree with the conjecture that our personalized classifier should perform better than interpolated discriminative classifiers. \n\nAt a high level, a generative approach is more advantageous in low-sample settings; but both will approach a similar accuracy if the local data volume is sufficient. Importantly, this is assuming they are using the same feature extractor parameters. \n\nHowever, we would like to point out that our generative modeling approach influences not only the formulation of the final personalized classifiers but also the process of global representation learning. \n\nIn pFedFDA, clients train the feature extractor to minimize the cross-entropy loss of a generative classifier using global feature statistics (refer to Section 4.2). Intuitively, this loss pulls client features towards the global feature distribution. This allows clients to benefit more from model interpolation, as there is less bias incurred through incorporating global knowledge (i.e., we can use a smaller $\\beta$ in Theorem 1.). \n\nIf an alternative method using interpolated discriminative classifiers does not employ additional regularization to guide the representation learning -  the performance may indeed drop below FedAvgFT on certain benchmarks. We can see a concrete example of this in the ablation study of FedPAC [6].\n\n[6] Personalized Federated Learning with Feature Alignment and Classifier Collaboration (Xu et al. ICLR 2023)\n\n**Q6 (Selecting a Preferred Method for Varying Non-IID Settings)** While dynamically selecting the personalization algorithm is interesting, it may be challenging in practice as the global feature extractor and estimated distributions evolve throughout training.\n\nWithout prior knowledge of client distributions, we think it makes sense to adopt pFedFDA as a generic one-size-fits-all approach, as it is robust to covariate shift, data scarcity, and is more efficient than FedPAC. \n\nIf additional resources are available, it may be reasonable to adopt a hybrid approach and interpolate the pFedFDA classifiers across all clients in the style of FedPAC after training has concluded.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NCqi2jTWWz", "reviewer": "Reviewer_WK31", "summary": "This work introduces pFedFDA, a novel approach to personalized federated learning that conceptualizes global representation learning as a generative modeling task. Specifically, the method involves shared representation learning, guided by a generative classifier characterized by a low-variance global probability density. Additionally, it iteratively refines global parameters and employs a local-global interpolation technique to tailor these estimates to individual client distributions. The authors validate the effectiveness of the proposed solution through experiments on benchmark datasets, demonstrating its robustness and applicability.", "questions": "please check \"Weaknesses\". I like the idea of this work and I am looking forward to the responses from authors.", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents an interesting and meaningful idea, complemented by a theoretical analysis of the bound on high probability estimation errors for the interpolated mean estimate, offering valuable insights and inspiration to the community. \n\nThe writing is clear and straightforward, with well-designed figures that effectively support and clarify the presented concepts.", "weaknesses": "There are several concerns that need addressing:\n1.In some cases, the performance improvements appear minimal. For instance, Table 2 shows that pFedFDA is outperformed significantly by FedBABU and pFedME.\n\n2.In Table 3, the authors employ a Dirichlet distribution (Dir(0.5)) for experiments under extreme data scarcity. It would be beneficial to lower the Dirichlet value, perhaps to 0.1, to conduct a more extensive evaluation under greater data scarcity.\n\n3.Table 5 indicates that the runtime of pFedFDA is longer than several other methods, including FedAvg, Ditto, and FedRoD, which calls into question the efficiency of the proposed solution.\n\n4.\tIn Equations 3 and 5, should there be a superscript ‘c’ on Σ to clarify the notation further?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1 (Performance Comparison of pFedFDA with Prior Works)** We believe this might be a misread of our results. While the performance improvements are not as significant in Tab. 2, pFedFDA is still competitive (achieving top-2 performance in 5 of the 7 scenarios). In particular, it beats FedBABU in all the 7 scenarios and beats pFedMe in 4 of the 7 scenarios. For the other 3 scenarios, pFedMe outperforms pFedFDA by .002, .004, .004, whereas pFedFDA outperforms pFedMe in the 4 scenarios by .002, .023, .03, .074.   \nIn the two columns in which our method is not within the top-2, CIFAR-10 Dir(0.1) and TinyImageNet Dir(0.1), we observe that the gap between all methods is smaller. This can be explained by the relative ease of the personalization tasks under those two scenarios. Notably, these scenarios have limited covariate shift between clients and the local data volumes are large enough for clients to build strong models even without collaboration.\n\nIf we consider these results alongside Tab. 1 and Tab. 3, our experiments indicate that pFedFDA is promising as a general strategy for real-world PFL settings where we may encounter challenges of covariate shift and data scarcity.\n\n**W2 (Setup for Data-Scarce Experiments)** Thank you for your suggestion. We want to emphasize that in this experiment, the Dirichlet distribution governs only class imbalance and not data scarcity. The extreme data scarcity setup in Tab. 3 is similar to what is considered in [1], where each client is assigned exactly one mini-batch of samples. Thus, changing the parameter of the Dirichlet distribution will not change the extent of data scarcity. We will clarify this setup in the camera-ready version of the paper if accepted.\n\n[1] Personalized Federated Learning with Feature Alignment and Classifier Collaboration (Xu et al. ICLR 2023)\n\n**W3 (On the Runtime of pFedFDA)** We do observe that there is a common computational cost associated with interpolation-based methods, as the local training takes additional steps to update each interpolated model directly (APFL), or optimize the interpolation weight between candidate models (FedPAC and our pFedFDA). \n\nCompared to [1], our interpolation task is more efficient, as we combine local and global models, rather than the models of each client.\n\nTo understand the overhead of pFedFDA in greater depth, we conduct an analysis of the run-time associated with each component of local training in Tab. 2 of the attached rebuttal document. Notably, our optimization of the interpolation parameters is responsible for most of the additional training time. In Tab. 3 of the attached rebuttal document, we compare the accuracy of our method when updating the interpolation parameter every $T$ rounds instead of every local update. While updating the interpolation coefficients every round results in the best performance, selecting $T=2$ or $T=3$ results in comparable average accuracy, thus this strategy may be preferable in resource-constrained settings to improve the efficiency of our method.\n\n**W4 (On Notation Clarity in Equations 3 and 5)** Thank you very much for your careful reading and detailed comments. This superscript is not required, as all classes share the same covariance matrix in our generative model (Line 178-179). Notably, this tied covariance assumption results in a linear decision boundary, allowing us to make a direct comparison to current works without altering the model architecture. We will make an effort to clarify this point in the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KjnkhSOhqD", "reviewer": "Reviewer_Qz6b", "summary": "Federated Learning (FL) combines data from multiple clients to train a global model but struggles with heterogeneous data. Personalized Federated Learning (PFL) creates individual models for each client, addressing this issue. Traditional methods face challenges with bias-variance trade-offs and rely on limited local data or costly techniques. This paper proposes pFedFDA, an algorithm that frames global representation learning as a generative modeling task. It adapts global generative classifiers to local feature distributions, improving performance in data-scarce settings.", "questions": "1. \"where Φ and H are the feasible sets of neural network and classifier parameters, respectively.\", in this context, it looks better to replace \"neural network\" to \"shared backbone\", do you think so?\n2. The notaions sometimes looks confused, which could be improved.\n3. In table 5, it should be pFedFDA not pFedDFA, right?\n4. The used dataset looks simple, could you use more complex dataset, such as cityscpaes, camvid, kitti, imagenet, etc to veriry your contributions?", "limitations": "the proposed method is over-reliance on the local distribution, so it is difficult to handle following cases: 1. a new client is added into the system and its local distribution is far away the original global distribution and the original distributions of all clients; 2. the local distribution of all clients are depondent on time (i.e., dynamic)", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. To implement personalized federated learing from generative view looks interesting and promising.\n2. This paper is well-structured entirely, but the notations could be improved.\n3. The mathematical proof looks good and sufficient.", "weaknesses": "1. Your method looks similar to the traditional personalzed federated learning. just as the algorithm 1, the shared backbone is just the weighted summation of the corresponding part in all involved clients. \n2. You mentioned that your shared backbone are trained in a generative way, but I can not get the core of of it. Actually, there are some other works[1] use the generative models (e.g., autoregression) as backbone to implement personalized federated learning, what is the difference between your methods and such baselines? \n3. Your global distribution parameters are weighted summation of all clients, so in my opinion, the global distribution may be bias due to data size skewness. There are many ways to overcome this, such as [2], etc. do you think such methods can be adopted by you to fix this issue of data skewness?\n\n[1] Kou, W.B., Lin, Q., Tang, M., Xu, S., Ye, R., Leng, Y., Wang, S., Chen, Z., Zhu, G. and Wu, Y.C., 2024. pFedLVM: A Large Vision Model (LVM)-Driven and Latent Feature-Based Personalized Federated Learning Framework in Autonomous Driving. arXiv preprint arXiv:2405.04146.\n[2] Kou, W.B., Lin, Q., Tang, M., Wang, S., Zhu, G. and Wu, Y.C., 2024. FedRC: A Rapid-Converged Hierarchical Federated Learning Framework in Street Scene Semantic Understanding. arXiv preprint arXiv:2407.01103.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1 (Distinction with Prior PFL Methods)** Our method falls under the personalization framework of shared representation learning with personalized classifiers (discussed in our related work, L85-107).\n\nThe distinction of our work is in our formulation of generative client classifiers. Notably, the formulation of the classification layer not only defines the final personalized model of each client, but also controls how the shared backbone parameters are trained. We further discuss the advantages of our generative classifiers in our response to W2 below.\n\n**W2 (Clarification of Generative Approach)** In our work, the term generative refers to the construction of our client classifiers, which we obtain through Bayes' rule and an estimate of the joint distribution of latent representations and class labels, $p(z, y)$. This is in contrast to prior representation-learning-based PFL methods which learn linear classifiers in a discriminative manner, learning $p(y|z)$ directly.  \n\nThe selection of client classifiers is important not only in obtaining an accurate personalized model but also in shaping how the shared backbone is trained. Using a generative classifier on each client to train the shared backbone, we can promote alignment in client features $p(z|y)$ while accounting for differences in class distributions $p(y)$. \n\nTo provide one final perspective on training the backbone with generative classifiers, by minimizing the cross-entropy error in model predictions, we are training the shared backbone to extract features according to the distribution defined by our generative model.\n\nIn your first provided reference, the backbone is trained using a input-level generative task, either autoregressive next-pixel prediction or masked pixel prediction. This technique of training large vision models is known as generative pre-training. We will edit our paper accordingly to make sure the usage of the generative terminology is clear. \n\n**W3 (Data Skewness in Generative Modelling)** Indeed, the weighted aggregation of parameters in most FL algorithms can lead to a bias towards clients with more local data. However, this update rule is still widely used in many PFL settings (e.g., in FedRep/BABU/PAC/FedAvg to aggregate neural network params), and an unweighted aggregation may increase the influence of noisy estimates from data-scarce clients.\n\nIn our work, we address the bias of using the shared distribution estimates through a local-global interpolated estimate. This has some similarities to the second provided reference, which uses the relative Bhattacharyya distance between RGB distributions to determine aggregation weights. While a similar approach could be used to obtain our interpolation coefficient, we optimize this coefficient directly to maximize personalized model accuracy.\n\n**Q1/Q2/Q3 (Method Presentation)** We appreciate the feedback on the presentation of our method, and we will incorporate these comments to improve the clarity of our work.\n\n**Q4 (Benchmark Datasets)** The selected vision datasets (CIFAR10/100, EMNIST) have been adopted by many recent works [1, 2, 3] to measure the effect of data heterogeneity on PFL tasks. We have additionally leveraged a set of natural data corruptions [4] for our CIFAR benchmarks, to simulate the complexity introduced by real-world covariate shift. We include TinyImageNet as an additional reference point in a large-data setting, but we focus primarily on the more challenging scenarios involving data scarcity and covariate shift. We leave the extension of our method to semantic segmentation tasks to future work. \n\n[1] Exploiting Shared Representations for Personalized Federated Learning (Collins et al., ICML 2021)\n\n[2] FedBABU: Towards Enhanced Representation for Federated Image Classification (Oh et al., ICLR 2022)\n\n[3] Personalized Federated Learning with Feature Alignment and Classifier Collaboration (Xu et al., ICLR 2023)\n\n[4] Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations (Hendrycks et al., ICLR 2019)\n\n**L1/L2 (Over-Reliance on Local Distribution Estimate)** While our method utilizes the local distribution estimate in generating personalized models, our personalized classiifers are based on an interpolated distribution estimate, which enables them to leverage global knowledge to reduce the variance in their local classifier. Additionally, we note that recent representation-learning based methods [1][2] only use local client data to estimate client classifiers, and based on our results in Tab. 3, we observe that our method is more robust to extreme data scarcity where the local distribution estimate is most limited.\n\nIn response to your concern on new-client generalization, we run the following experiment, which can be found in Tab. 1 of the rebuttal PDF: \n\n- We train each method on CIFAR10 Dir(0.5) using half of the total clients throughout training. At test time, we evaluate on these clients, as well as the second half of clients not seen at training time. We additionally evaluate new-client generalization performance under covariate shift, by corrupting the images of all new clients with each of the 10 corruptions considered in our paper.\n- We observe that our new-client generalization performance is superior than the baseline FedAvgFT for all settings with and without covariate shift.\n\nRegarding the compatibility of pFedFDA to dynamic client data distributions, we think this is an interesting direction for future work, but in this paper we follow the experimental setup of our cited baselines in assuming that client data distributions static.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank the reviewers for their detailed comments and feedback. We will revise the paper accordingly to further clarify our work and address the points brought up in these discussions.

In our attached rebuttal PDF, we have provided the following additional experimental results:

Tab. 1: Evaluation of method generalization to clients unseen at training (in response to reviewers Qz6b and BoVr).

Tab. 2: Analysis of system run-time corresponding to each component of local training (in response to reviewer 3xQF). 

Tab. 3: Evaluation of pFedFDA with intermittent updates of the interpolation parameter $\beta$ (in response to reviewer 3xQF and WK31).

Meta-review: The paper proposes a personalized FL method that uses a generative classifier by considering the bias-variance trade-off. The paper is well-written, with a clear structure that logically progresses from the introduction to the methodology and results. In the rebuttal, all the reviewers are satisfied with this rebuttal. I recommend acceptance for this submission.
Decision: Accept (poster)