FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts
Authors: Dario Fenoglio
Mohan Li
Pietro Barbiero
Nicholas D. Lane
Marc Langheinrich
Martin Gjoreski
Venue: NeurIPS 2025 poster ()
Keywords: federated learning
clustered federated learning
privacy
distribution shifts
scalability
efficiency
unsupervised clustering
test-time adaptation
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: FLUX is a clustering-based Federated Learning framework that handles non-IID data without prior knowledge of distribution shifts, achieving state-of-the-art performance with minimal overhead.

Abstract: Federated Learning (FL) enables collaborative model training across multiple clients while preserving data privacy. Traditional FL methods often use a global model to fit all clients, assuming that clients' data are independent and identically distributed (IID). However, when this assumption does not hold, the global model accuracy may drop significantly, limiting FL applicability in real-world scenarios. To address this gap, we propose FLUX, a novel clustering-based FL (CFL) framework that addresses the four most common types of distribution shifts during both training and test time. To this end, FLUX leverages privacy-preserving client-side descriptor extraction and unsupervised clustering to ensure robust performance and scalability across varying levels and types of distribution shifts. Unlike existing CFL methods addressing non-IID client distribution shifts, FLUX i) does not require any prior knowledge of the types of distribution shifts or the number of client clusters, and ii) supports test-time adaptation, enabling unseen and unlabeled clients to benefit from the most suitable cluster-specific models. Extensive experiments across four standard benchmarks, two real-world datasets and ten state-of-the-art baselines show that FLUX improves performance and stability under diverse distribution shifts—achieving an average accuracy gain of up to 23 percentage points over the best-performing baselines—while maintaining computational and communication overhead comparable to FedAvg.

FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts
Dario Fenoglio" Università della Svizzera italiana Lugano, Switzerland
Mohan Li" Università della Svizzera italiana Lugano, Switzerland mohan... 1 Licusi.. ch
dario. d iouusi..ii.... ch
Pietro Barbiero
Nicholas D. Lane University of Cambridge Cambridge, United Kingdom nd1320cam. ac uk
IBM Research
Zurich, Switzerland pietro. barbiero@ibm.m.mmmmmm.. com
Marc Langheinrich Università della Svizzera italiana Lugano, Switzerland marc lerrheenngeiiiiccco www.siicchhii...o. ch
Martin Gjoreski Università della Svizzera italiana Lugano, Switzerland martin. serriiingorresiiuueeee
Abstract
multiple clients while preserving data privacy. Traditional FL methods often use a global model to fit all clients, assuming that clients' data independent and identically distributed (IID). However, when this assumption does not hold, the global model accuracy may drop significantly, limiting FL applicability in real-world scenarios. To address this gap, we propose FLUX, a novel clustering- FL (CFL) frame- work that addresses the four most common types of distribution shifts during both training and test time. To this end, FLUX leverages descriptor extraction and unsupervised clustering to ensure robust performance and scalability across varying levels and types of distribution shifts. Unlike existing CFL methods addressing non-IID client distribution shifts, FLUX i) does not require any prior knowledge of the types of distribution shifts or the number of client clus- ters, and ii) supports test-time adaptation, enabling unseen and unlabeled clients to benefit from the most suitable cluster-specific models. Extensive experiments across four standard benchmarks, two real-world datasets and ten state-of-the- art baselines show that FLUX improves performance and stability under diverse distribution seffss-achieving an average accuracy gain of up to 23 percentage points the best-performing baselines-ar while maintaining computational and communication overhead comparable to FedAvg.
Federated Learning (FL) enables collaborative model training
prreeervng client-side
Introduction
Federated Learning (FL) [1-4] is a distributed and preeey-pres Machine Learning (ML) paradigm that enables multiple isolated clients collaboratively train models without sharing their private local data. Traditional FL methods often use global model to fit all clients' data [1, 5, 6]. assuming that clients' data are independent and identically distributed (IID). However, this assumption
'Equal contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

PLYIX
PXXNN) shif
Fosture spoce pils)
+: Clizet A Cliiittt B Blue color: Clas
Onngg color: Clas
Gray oilor. Festue spact mon
FDE: Protubblity Danely Fundion
Figure 1: Types of data distribution shifts. (a) Feature distribution shift: two subsets differ in feature similar. (h) Label distribution shift: two subsets differ in
distributions, while label distributions label distributions, while feature distributions (for each class) are similar. (c) P(Y|X) concept shift: two subsets share the same feature distributions but differ in label distributions. (d) P(X concept shift: two subsets share the same label distributions but differ in feature distributions.
rarely holds in practical FL scenarios, where clients often exhibit distribution shifts due holding non-IID data" [7. 8). Such heterogeneity can significantly degrade the performance of the global model, limiting the effectiveness of FL in real-world applications [9].
To address this problem, both Clustered Federated Learning (CFL) [10, 11] and Personalized Feder- ated Learning (PFL) [12, 13] have been proposed. These approaches relax the global model constraint by adapting models to subsets of clients with homogeneous data distributions-via clustering in CFL or client-specific personalization in PFL. Despite their promise, existing methods address only specific forms of heterogeneity (e.g.. feature or label shift), and often fail when multiple or unforeseen shifts occur [12-17]. Many rely impractical assumptions, such prior knowledge of the optimal number of clusters [11, 15, 16, 18), and incur high communication and computa- tional overhead [13, 14, 17, 18). Crucially, most methods assume that test-time clients have already participated in training and contributed labeled data [11-19]. This assumption breaks down in realistic deployments, where unseen and unlaheled clients may appear post-training with unknown data distributions. To overcome this limitation, Test-Time Adaptive FL (TTA-FL) methods have emerged adapt pre-trained models to the distribution of test-time clients without requiring labeled data [19, 20J. However, they typically assume a single starting global model and rely on online adaptation procedures or extra interactions with the client, limiting their practicality and robustness in deployment.
To address these challenges, we introduce FLUX (Federated Learning with Scalable Unsupervised Clustering and Test-Time Adaptation), a novel clustering- approach that efficiently handles data heterogeneity in FL. under minimal assumptions. Our key contributions are:
We propose FLUX to address four common types of data distribution shifts in FL: feature shifts, label shifts, P(Y ( J-concept shifts, and "(X(())coneee) shifts. Unlike most existing frameworks, FLUX does not require prior knowledge of unseen data distributions or the number of clusters.
We empirically demonstrate that FLUX supports test-time adaptation by assigning previously unseen and unlabeled clients to the most suitable cluster models. FLUX consistently outperforms 10 state-of toitte-ofthhe-eee the-art (SOTA) baselines- -inrluding CFL, PFL, and TTA-FL -LL- across four standard FL benchmarks and two real-world datasets. Evaluations span a broad range of scenarios, covering the four most common types of distribution shift, their combinations, and eight levels of heterogeneity severity (from none to extremely high).
We provide both theoretical bounds and empirical evidence that FLUX incurs minimal computational and communication overhead compared to baselines and verify its scalability, enabling deployment across a large number of clients while ensuring privacy guarantees.
To the best of our knowledge, FLUX is the first scalable and general- FL framework explicitly designed to address the four most common distribution shifts during both training and test time.
2 Background
Traditional FL under IID assumption. Traditional FL systems [1] consist of K E N clients, denoted by KC 21, 2..., K}. coordinated by central server to collaboratively train an ML model while preserving data privacy. Each client KC holds private dataset (zx()).g) (y())) a realization
We refer these clients non-IID clients in the next sections.

---

of the random variables (x (4) drawn from the data distribution P(X y (()). Under IID assumptions, FL assumes IID data across clients, i.e., P( = = Y) for all k. Each client holds (A) E N samples, with (:(k) <TR= x denoting feature vectors and ( ε {0, 1) "x xu the corresponding labels, where is the number of features and the number of classes. In each round, clients independently update local parameters g(k) E ((() C IPP by minimizing a local loss on their private data, with the number of parameters. After local training, clients send their updated parameters to the server, which aggregates them using permutationninvariann method, typically FedAvg [1]. The aggregated global model 9 is broadcast back to clients to initialize the next round, and the process repeats until convergence 0", minimizing the overall risk across client distributions:
1)
8r arg max
Σ
log P(y x; 0)
(I.))E(2(())
Distribution shifts. Real-world FL tasks involve clients with combination of distribution shifts, driven by factors such as geographic and demographic variations, or even adversarial attacks. While distribution shift has been studied extensively in centralized settings [21-24], it remains a relatively new challenge in FL. The FL community has developed numerous approaches- as FedNova [25], FedProx [5], and FedDyn [26]-10 mitigate data heterogeneity, but these methods typically address only specific aspects of the non-IID challenge. This limitation is further exacerbated by the fact that the server has limited knowledge and control over client data distributions due to data isolation. Below, we outline four typical types of FL distribution shifts, as illustrated in Figure I:
Feature distribution shift: marginal distributions P(X) vary across clients.
Label distribution shift: marginal distributions P(Y) vary across clients.
Concept shift (same features, different label): conditional distributions P(Y]X) vary across clients. Concept shift (same label, different features): conditional distributions P(X vary across clients.
Unlike centralized training, FL. clients, particularly in cross-device scenarios, often operate on devices with constrained computational resources. These limitations restrict both the computational capacity for training and the complexity of deployable models, reducing their ability to capture diverse and complex data distributions across clients [14, 16]. As a result, the trained models may lack sufficient expressiveness, adversely affecting both performance and generalizability [14].
Clustered Federated Learning. To address distribution shifts in federated environments, CFL extends traditional FL by partitioning the overall training data across clients into M E N distinct clusters. Each cluster is associated with a unique data distribution P(X Ym) such that all data within that cluster are drawn from the same distribution. Let C = (c(1), c(2) c(K) denote the set of Justen-ts vectors, where each c(N) E IR M represents the degree of membership of client to each of M clusters, and satisfies > Cm
(k) 1. Based on the nature of cluster assignments, we define two CFL frameworks: soft-CFL and hard-CFL (see Appendix A. for more details). In soft-CFL, the vectors of c(k) are fractional values in [0, 11, reflecting probabilistic membership across clusters. In contrast, hard-CFL requires that each (kk) be one-hot vector, i.c., c(R) {0,1) 1M meaning that client is assigned exclusively to a single cluster. Under hard-CFI., the client set IC is partitioned into disjoint subsets corresponding to the different clusters, and within each cluster m, traditional FL. is employed to optimize a dedicated model 6% ensuring that the conventional FL assumptions hold locally for the data drawn from P(Xms Ym). Formally, CFL seeks to jointly optimize the model parameters and the cluster assignments as follows:
K Σ
M
(0 1M
{c(k)= 1
Σ
(K) log P(y 2; x;8m) (2)
[k=1
arg
max
{8 1M
(2()) (()))
3 CFL Challenges and Problem Definition
I Limitations and challenges in current CFL approaches. As summarized in Table 2, existing CFL methods fail to satisfy several key FL requirements. Most lack a robust mechanism for assigning clusters at test time- eime-casycallly for unlabeled clients-an are not evaluated across all four types
3

---

of non-IID shifts. Moreover, methods such as [15, 11] require prior knowledge of the number of distributions or clusters before initiating FL training (i.e., M.), which is often impractical in real-world applications. Additionally, most approaches impose significant computational burdens, either on the server side [14] or the client side [17], and demand increased communication costs multiple models are transmitted [11]. These limitations hinder scalability, particularly as the number of clients or potential clusters grows, a common scenario in real FL deployments. For this reason, we opted for hard-CFL approach: unlike soft-CFL methods, which require maintaining multiple models per client and learning personalized weight vectors (hindering efficiency and generalization to unseen clients), hard clustering provides more scalable and deployable solution. A more detailed comparison among our work and baselines is provided in Appendix E.2.
Problem statement. We consider an FL setting with K clients. Each client k holds a private dataset (1z(k) y)(h) drawn from one of M unknown data distributions {P(Xm, Ym) "m)mm= where 1 S. M I K. Our objective to (i) determine the number of clusters M, (ii) identify the cluster assignment C, so that all clients in cluster m have (approximately) IID data from the same distribution P(Xm. Y.), and (iii) optimize the model 8n for each cluster m. At test time, our goal is to assign any newly arriving. unlabeled client to the best-fitting cluster-specit model 6n Crucially, no prior knowledge of the underlying data distributions or the number of distribution shifts is required; both M and C are learned from the clients' data. The methodology must ensure that the solution is as scalable as traditional FedAvg and maintains the same privacy level.
4 FLUX
In this work, we propose FLUX, a robust, computationally efficient, and scalable CFL framework that handles all four types of data distribution shifts without requiring prior knowledge of client data. This section introduces the theoretical foundations of FLUX, using a probabilistic graphical model represent the relationships among client distributions, descriptors, and cluster assignments, enabling decomposition into independently optimizable objectives (Section 4.1). We then present the operational pipeline of FLUXX, designed for adaptability and real-world utility (Section 4.2).
4.1 Probabilistic Modeling and Optimization Objectives
FLUX aims to predict (k) from (*) using a cluster-specific model parameterized by O... where On E @n depends on the cluster m to which client k is assigned. During training, each client learns local model ((x) E (R) based on its private dataset. From these data, then constructs a compact de- scriptor d(k) E (K) via a feature extractor á pa- rameterized by 1 E V which is designed to capture the essential information of the client's data distri- bution. After collecting the descriptors {d(() y
Figure 2: FLUX's PGM. Solid line: data gener- ating mechanism. Dashed line: inference direc- tion. Gray line: present only during training.
from all clients, the server applies an unsupervised clustering algorithm U, parameterized by A E A. to obtain the cluster assignments [ These assignments allow the aggregation of local models into cluster-specific models 0m, each optimized to generalize well across clients in cluster m.
We formalize FLUX as probabilistic graphical model (PGM) capturing dependencies among variables. Each client & contributes to the joint distribution via its data (x) (*)), descriptor ((*),
and cluster assignment C(k), Figure 2 illustrates the global structure, which facctoriees per client as:
P(C D(*), (M) (1), (()) (), ((D))) ((M); :A) :) P(
v) ((Y (+) (+))
clustering
descriptoc extractor
local classifier
Each element in the decomposition represents a distinct component of FLUX:
(k) (())) (local classifier): Each client & models its joint distribution, which factor- izes as ((y(k)) ( ((X())). ((()))(())))))) Since (()()) is independent of ((k), each client locally
learns the predictive model by minimizing the corresponding negative conditional likelioiddood.

---

P(Y ((())) (D)) (descriptor extractor): A feature extractor & with parameters V maps the high-dimensional data (Y (k) to a compact descriptor D At test time, only (x)k is available, i.e. P(D(k) (*, V). Parameters Vi are learned on the client side (Section 4.2.1).
P(D(*) [C()),A (unsupervised clustering): This term models the probabilistic assignment of clients to clusters based on their extracted descriptors. It parameterized by an unsupervised clustering algorithm 2/ with parameters A (see Section 4.2.1).
After clustering, the server aggregates local models into cluster-specific parameters 2n by determin- istically combining assignments {C(k) } with {O(k), This hierarchical decomposition disentangles the overall optimization into distinct sub-problems problemss-clusteringgg descriptor extraction, and local classification each of which can be optimized independently as follows:
K
{g(K).a) (hm
1k-1. X arg
max
Σ
log P(dtk) |
((()
1000) LK
E
log | ;00)
E
log P x= (3)
+
(2.7) ((1))
4.2
FLUX Pipeline
Figure 3 provides overview of the FLUX pipeline, illustrating how its components translate theoret- ical models into actionable strategies. This subsection details the pipeline's mechanisms during both training and inference phases. During training (Section 4.2.1), FLUX securely extracts representative descriptors from client data which are then used to cluster clients with similar distributions in an unsupervised manner to accommodate data heterogeneity. During the inference phase (Section 4.2.2), FLUX assigns trained models to unseen clients by matching their data distributions to the closest clusters without requiring labeled data, ensuring adaptability in real-world scenarios.
Clustering Truining phnse
Inference phase
SIATL
Dneeriptor ectrarier (ن
Agaregation
Cleests with Let
+0 (X) g
Descripiar 0i (i:g))
(0x 0.0)
Discrittr
diyy (11) v)
sleass
Figure 3: Overview of the FLUX framework for efficient unsupervised CFL. FLUX operates without prior knowledge of client data, handling distribution shifts and optimizing cluster-specific model assignment to unseen, unlabeled clients at inference.
4.2.1 Training Phase
The training phase is designed to address all four types of data distribution shifts-feature distribution shifts, label distribution shifts, P )ccccccpppt shifts, and P(X|Y) shifts. To identify these shifts, extract representative descriptors that capture the key statistical characteristies of each client's 's labeled data, thus approximating the relevant data distributions for clustering.
Descriptor extractor. Consider set of M data distributions [P(X,, m. Ym)
J.M m=m1 where each client holds data sampled uniformly from one of the M distributions P(Xm. Y.... We define the descriptor
extractor: dbg R x(z+u) + RRL & (()) x (z + u)), parametrized by v., that maps local dataset (k) (k) P(Xms Ym) to a descriptor (k)) := (kz(k), (k),,, v)). For any two client pairs 1j and k2, the extractor must satisfy the requirements below:
(R1) Distribution fidelity. The function ø is designed to approximate reference distance D (e.g.. Jensen-Shannon
Shannon or Wasserstein distance) between the corresponding distributions:
(aa d 2|2 (( (())), ((a . (ka))) s
(4)
5

---

In other words, the extractor ø maps similar joint distributions to nearby descriptors and dissimilar ones distant descriptors.
(R2) Label agnosticism. A sub-vector d'(h) S ((()) must be computable without any labels:
d'(b) == ( (k) 0: e)) E R., p P
(5)
enabling descriptor extraction and similarity matching at test time. The sub-vector d'(k) encodes the marginal characteristics of the imput distribution based solely on the features x²(a), (R3) Compactness. The function & introduces minimal overhead relative to vanilla FL: its com- putational cost is negligible compared to single local training epoch, and the descriptor dimensionality L satisfies L < (typically d/p- I 10 10-2), ensuring that the additional commu- nication cost remains marginal relative to the model update size p.
Practical implementation. To capture all four shift types, we factorize the joint distribution as P) = P(Y P(X) and design & to encode the marginal P(X) and conditional P(Y X) separately. Appendix C.4 details how this design enables detection of feature, label, and both concept- shift variants, together with the step-by-step implementation. Concretely, each client's 's data x²(k is first mapped to latent space by the shared encoder fe RR R" with parameters 9, then compressed by a clienttinvariant reduction f-- R" +R' (v > I), parametrized by ei (e.g., a shared PCA fitted on synthetic reference points, with 1 = 10):
(6) To capture the marginal distribution P(X), we compute the first two moments of the reduced
26) =
latents-ment mean ( and covariance I(k) To capture the conditional distribution P(Y X), we
compute the class-conditional moments {uu) (k) > J'=1 The full descriptor d(k) E RR(U++) is
obtained by concatenating all marginal and conditional moments:
(k)) [m(e, EXA), Mi
(1) (A)
(k) (k) ..... MU 20
This instantiation meets every requirement: (R1) the mapping is provably Lipschizz-equivalent to the 2- 2-Wasserstein metric, with << 1.1 on MNIST and (and < 0.54 under Jensen-Shannon), (R2) the label-agnostic sub-vector d'(k) effectively captures P(X); (R3) it adds negligible compute and communication ratio L/p= 10 Theoretical justifications, motivation, and further details appear in Appendices C.I, C.3, and D.1. Additionally, differential privacy [27] can be seamlessly plugged into d(() without affecting FLUX accuracy (see Appendix C.2). An ablation study and pseudocode provided in Appendix F.6 and Algorithm 1.
Unsupervised clustering. In real-world FI. settings, prior knowledge client data distributions rarely available. To address this, we propose using an unsupervised clustering method that determines the proper number of clusters M and assigns each client its corresponding cluster based on their descriptors. Formally, given the set of client descriptors (d(*) K parametrized by X, the cluster assignments C are computed as:
k== and clustering algorithm 24
C C (d((d()) d(2) dd"2).....d (K);
(8)
In our implementation, we design an adaptive density-based clustering method that automatically determines the number of clusters present. Specifically, we extend DBSCAN by estimating the parameter through elbow detection on the sorted recond-ncarest neighbour distance curve, calibrating with -ataset-specific scaling factor, and reassigning noise points as singleton clusters to ensure every client is represented (see Appendix D.2 for details). Nonetheless, our formulation is agnostic to the choice of clustering method, provided it is unsupervised: in Appendix F.6.2, we report ablation studies with alternative clustering strategies, showing that our descriptors consistently enable robust client grouping regardless of the clustering algorithm, and Appendix D.3, we provide illustrative examples of how the clustering process operates in practice.
4.2.2 Inference Phase
During inference, newly joining test clients require access to trained models. As these clients lack labeled data, we propose mechanism that assigns test clients to the most suitable cluster-specific model based solely on their feature distributions. This approach optimizes inference process by leveraging the clustering structure learned during training.

---

Test-time cluster assignment. For each test client q, only feature- descriptors can be extracted, as label information is unavailable. By fulfilling (R2), the descriptor extractor & directly yields the label-agnostic descriptor as d' = x(x(*) 0; v)), which encodes the marginal distribution. Consequently, with information solely from the feature space P(X), ((Y| -conccetp shifts cannot be solved during test time, P(X) across clients identical (same input features, different labels).
R Each cluster m identified during training associated with a centroid Ym. computed as the mean of the sub-vectors d'(k) of all clients k in that cluster. To assign a test client y to a cluster, we then compare its deseriptor d'(4) with the cluster centroids 7' using similarity metric that measures proximity (e.g., Euclidean distance). The client assigned the cluster with the closest centroid according to the chosen metric:
Precisely, in our implementation. d'(s) = lus ( ]
e
= arg
min k(d"(n) -mm)
())
This assignment ensures that the client y uses the cluster-specific model 03 that is most representative of its feature distribution. We provide the pseudo-code for our implementation in Algorithm 2.
5 Results
This section presents the experimental setup and results from two primary scaling experiments, analyz- ing FLUX performance under varying non- levels (Section 5.2) and an increasing number of clients (Section 5.3). These experiments evaluate and compare the scalability, robustness, and adaptability of FLUX against SOTA baselines across diverse distribution shifts and client configurations.
5.1 Experiment Settings
Non-IID dataset generation. We use six publicly available datasets in our experiments: MNIST [28]. Fashion-MNIST (FMNIST) [29], CIFAR-IO, CiFAR-I00 [30]; and two real-world datasets, CheX- pert [31] and Office-Home [32]. To simulate non-IID conditions in FL, we employ ANDA, a publicly available toolkit that enables data operations such class isolation and label swapping. See Ap- pendix B. for details on the ANDA and the datasets partitioning strategy.
Baseline algorithms. We evaluate approach against ten basclines listed in Table 2, including Fe- dAvg [1], CFL [14), IFCA [15], FeSEM [18). FedEM [16), FedRC [11]. FedDrift [17], pFedMe [12]. APFL [13], and ATP [20]. Full baseline descriptions provided in Appendix E, experimental setup, hardware, models, and hyperparameter configurations are detailed in Appendices B.2 and B.3.
5.2 Robustness Across Different Types and Levels of Data Non-IID Heterogeneity
Superior performance of FLUX across varying heterogeneity levels and datasets. We benchmark FLUX against SOTA base- lines across four shift typestypeePtt P(X), P(Y). X)) and P(X each instantiated at eight increasing non-lII levels. Full setup and results are detailed in Appendix F.I. Fig- ure 4 reports the average accuracy across shift types MNIST (results for FMNIST, CIFAR-10, and CIFAR-100 are provided in the Appendix F.1), highlighting performance trends as heterogeneity severity increases. FLUX consistenty achieves robust performance across the heterogeneity spectrum, matching or exceeding PFL methods under the simplified known association condition cettiiion-whrreee test-time cluster assignments are assumed be known while maintaining high accuracy and stability even in the more realistic test phase, where such information is unavailable. Notably, FLUX yields absolute accuracy gains of up to 12.4, 23.0, 7.0, and 3.0 percentage points (pp) over the best-performing baselines on MNIST, FMNIST, CIFAR- 10, and CIFAR-100, respectively. Table summarizes the results for each dataset, averaging performance across all heterogeneity levels. In the test phase setting, FLUX consistently outperforms all baselines, improving accuracy by up 7.84, 11.86, 2.17,
Known Ascociation
Test Phose
1.0
0.8
Accuracy
0.6
Pealloo
Heterogencity
Lovel
Heserogeneicy
Level
Figure 4: Mean accuracy and standard deviation
across heterogeneity levels for MNIST.
7

---

and 1.5 pp over the best-performing baselines CCLiinnes---LLL-- on MNIST and FMNIST, IFCA on CIFAR-10, and FeSEM on CIFAR- C10, respectively. Even under the simplified known association setting, FLUX matches or surpasses the best methods on MNIST, FMNIST, and CIFAR-100. On CIFAR-10, FL.UX slightly trails PFL. methods expected, as they are fine-tuned on the same client distributions seen at expeetee
test time-but their performance drops sharply on unseen clients. By contrast, ATP-an an unsupervised test-time adaptation serthod-show instability and underperforms compared to FLUX. particularly on complex datasets like CI0, where adaptation often leads to overconfident but incorrect predictions. Furthermore, FLUX-prior, which applies K-means clustering with the true number of clusters M (oracle knowledge), sets new CFL benchmark by achieving the highest accuracy across datasets and evaluation conditions. is worth noting that most CFL baselines rely on this cluster information (Table 2), significantly simplifying their clustering process. Appendix F.3 further shows FL.XXS s robustness when multiple shift types occur simultaneously.
Robust performance on real-world datasets. To assess real-world applicability, we evaluate FLUX on the CheXpert dataset under three naturally occurring levels of non-IlI heterogeneity (see Appendix F.4 for details), and the Office- Hifeee-Hmee&o dataset under four inherent domains. As CheXpert is a multi-label classification task, we report macro-averaged ROC AUC. Results summarized in Table 1 show that FLUX consistently outperforms all baselines in both the known association (up to 8.5 pp over the best- pest-performing baseline, APFL) and test phase settings (up to 17.5 pp). On the Office-Home dataset, FLUX attains competitive performance, closely matching APFL in the known association setting and outperforming all baselines in the rest phase by up to 1.3 pp. In contrast, most metric- and parameter-based CFL methods collapse to single global model, failing to capture subtle real-world distribution shifts. FLUXX performance closely matches that of FLUUX-prior, underscoring the effectiveness of our learned descriptors and unsupervised clustering strategy- even without access to the true number of clusters.
Datasel
MNIST
FMNIST
CIFAR-10
CIFAR-100
CheXperl
Office-Home
Algorithm Known A.
Test Plase Known A.
Test Phase Known A. Test Phase Known A. Test Phase Known A. Test Phase Known A.
Test Plase
FodArg
$0
8..4211
66.5
6.7
3.9
3.9
38.0 23
$6.1 1.
17.1
009
IFCA
7.
78.245.4
724452
63
25
2.
4.1
29
58.504
328
7.
FedRC
4..5+5.1
4..9
54.3
55.
17.3+22
172224
338
37.9
588=04
583+04
22
22.2
FelEM
5.0
4..0+45
55.7
552
5..
130+22
175+24
1.8+111
1.
585+04
58
28
28
RSEM
8..813.1
8.78
71.9337
6.
26
35
34
19
390004
18
27
2.
FodDrift
91.1.129
80.7121
51.
25.1
32.1122
39.1311
26.5
616206
61.6106
19.5
36.14.11
CFL
$1.31
1.91
66.0
69.4
32242.1
332223
1.
18.6
58
5..5
313333
2..0
PFedMe
9.35
N/A
11
NIA
42.4 1.1
N/A
36.1
NAA
09.4
N/A
30
N/A
APPL.
9:3
84.7+24
81.9
09:228
1.
366+11
4..2
37.3 03
7233002
640+04
435+11
36.7+0.4
ATP
N/A
85.6+188
NIA
68
NAA
33
N/A
3.55
NIA
N/A
N/A
37.9
FLUX
2.9
94.0 2.2
79
8.2
23
3.
20.7
4133
79.4
39.2 0.
39.2 0.3
FLUX-prior
9,
95.7
1.7
11
14
40.1
1.6
3.1
428
41.3332
7.3
0.9
1..2
18.9
Table 1: Overall performance on MNIST, FMNIST, CIFAR-10, CIFAR-IOO, Office-Home (accuracy), and CheXpert (ROC AUC). Known A.: Known Association. N/A: Not available.
Known Association
Test Plaze
Training Time
1.0
1.0
2
0.8
0.8
Time (scconds)
Ps530
224
Accuracy
Accuracy
0.6
200
OPF
APPL
0.4
0.4
FLOX
AT-27L
0.2
0.2
50
100
25
50
1000
225
50
100
Number of clients
Number clients
Numbbr of clients
Figure 5: Mean accuracy and standard deviation on MNIST dataset with varying numbers of clients. Left: known association condition, where test-time cluster associations are available. Middle: test phase condition, where cluster associations are inferred. Right: training time per 10 rounds.
8

---

ALGORITHM
CAT.
ALL
TEST
No CLUSTER
Low COMM.
LOw COMP COST (SERVER)
Low COMP COST (CLIENT)
SCALABILITY
N9S-IID
ADAPTATION
NUMBER PRIOR
COST
FEDAVG [1]
NIA
NIA
N/A
NIA
IPCA [15]
CFL.
FESEM [18]
CFL
FEDEM [16]
FEDRC [11]
PEDDRIFT [17]
CFL
CFL [14]
CFL
PREDME (12)
PFL
APEL [13]
PFL
ATP [20]
AFL
FLUX
CFL
Table 2: Qualitative comparison of FLUX and baselines. Cat.: FL category. All non-lID: designed to tackle all four non-IID types. Test adaptation: adapt to unseen, unlabeled clients. No cluster number prior: not required knowledge of distribution numbers. Low comm. cost: relatively low communication cost (comparable to FedAvg). Low comp. cost (Server/Client): relatively low computational cost on server/client side (comparable to FedAvg). Scalability: scales efficiently with large client numbers. N/A: not applicable. v: property satisfied. O: property conditionally satisfied.
5.3 Scalability and Efficiency Across Increasing Numbers of Clients
FL is designed to jointly train models across millions of clients with limited computational resources and memory. This necessitates efficient and lightweight methods capable of handling numerous data distribution shifts (i.e., clusters). To evaluate scalability, we measure the accuracy and training time of FLUX and the baseline methods by increasing the number of clients from 5 to 100 (Figure 5). Due to prohibitive memory and computational costs (see Appendix B.3), FedDrift and CFL could not be evaluated with 100 clients. Detailed experimental setup and results are provided in Appendix F.2.
FLUX maintains high accuracy with increasing numbers of clients and clusters, demonstrating adaptability to large-scale FL. Figure 5 shows that accuracy generally decreases for all methods, confirming that as the number of clients-an and therefore clusters ceattess-nnrrassee clustering becomes more challenging due to the presence of numerous diverse data distributions, reduced training data per cluster, and the higher aggregation divergence inherent to FL. Despite this, FLUX and FLUX- prior maintain consistent accuracy in both evaluation settings (known association and test phase), outperforming baselines, which exhibit significant accuracy degradation as the number of clients increases. During the test phase, FLUX sustains consistent accuracy above 84%, whereas the closest baseline, APFL, declines to over 70%.
FLUXX demonstrates scalability and efficiency with minimal computational overhead. We further evaluated the efficiency of FLUX by measuring the overall training time as the number of clients increased. As illustrated in Figure 5, FL.UX shows training times comparable to FedAvg, with only minor differences in execution time (on the order of seconds). The efficiency of FLUX stems from its minimal modifications to the traditional FedAvg framework. Specifically, the additional costs introduced by FLUX- -aiuariiirrlly the communication step and clustering process-are negligible relative to the total costs of model parameter transmission and aggregation. The communication overhead is proportional to the length of the descriptor (L). which is significantly smaller than the total model size (L/p < 3.5 x 10 10-3). Similarly, the clustering cost (O(L (o)( is substantially lower than the aggregation cost (O( 0)). Notably, the computational time of the fastest CFL baseline, FeSEM, is more than 4 times that of FLUX, while the slowest, FedDrift, requires over 300 times the computational time of FLUX. Further details are provided in Appendix E.2.
6 Discussion and Related Works
Clustered Federated Learning. CFL partitions the overall client data into clusters based on training behaviors or data distributions, serving as a middle ground between PFL and traditional FL. By aggregating within clusters, CFL can potentially accelerate model convergence and mitigate local overfitting. CFL methods are typically categorized as hard- [14, 15, 17, 18] or soft-CFL [11, 16], and further distinguished by their clustering principles: metric-based [15-17, 33, 34] or parameter- bused [11, 14, 18, 35]. Metric-based CFI. (e.g., base-ba may miscluster clients with similar empirical risks but divergent loss distributions, showing that single loss value is insufficient for reliable clustering. Likewise, feature-based metrics such as frequency coefficients [33] cannot

---

distinguish clients with different conditional distributions but identical marginals. Parameter- based CFL may misgroup models due to permutation invariance and overparameterizzaton,, assigning identical functions with different parameters to separate clusters or vice versa [36]. In contrast, F1.UX introduces a descriptor-based clustering paradigm, where descriptors approximate the 2- 2-Wasserstein distance and jointly capture both marginal and conditional properties of client data, capability essential for handling all four types of distribution shift simultancously.
Personalized Federated Learning. PFL [10, 13, 37-53] reframes FL as a client-centric opti- mization. learning a distinct model for each client to address data heterogeneity. PFL methods include fine-tuning shared global model via additional local updates or meta-learning [10, 37, 38]; model-decoupling approaches that split networks into shared and client-specific components or adapt batch-norm layers [13, 30--99-441 and regularization based methods that add proximal or penalty terms linking personalized and global parameters [10, 54, 55]. While effective with ample labeled data, PFI. demands hyperparameter tuning, adds on-device compute and memory overhead, and and-being inherently supervised cannot tackle unseen or unlabeled clients or distribution shifts at test time.
Test-time Adaptive FL. TTA-FL tackles post-deployment distribution shifts by adapting a pre- trained global model to each client's unlabeled test data using unsupervised objectives-e.g, entropy minimization [19, 20, 56] or contrastive losses [57, 58]. Without labels, these objectives are ill- conditioned: entropy minimization often yields overconfident mispredictions under concept shift, and contrastive losses can collapse on small or imbalanced batches. To enhance stability. recent works restrict adaptation few parameters (e.g., interpolation weights) [19]. However, they still depend on supervised pre-training of personalized components, rendering them unsuitable for truly test-only clients. A more detailed discussion of these methods is provided Appendix A.1.2.
Limitations and Future Works
FLUX has two primary limitations. First, to obtain statistically robust descriptors of client data distributions, a sufficient amount of diverse training data is required, with the exact quantity depending on the variability of the underlying data distributions. Second, FLUX operates one-shot CFL framework, which does not account for dynamic scenarios where client data distributions evolve over time (i.e., distribution drift). Nonetheless, the efficiency and simplicity of FLUX make it well-suited for dynamic implementations. The clustering process can be repeated whenever a drift occurs within subset of clients, with the new clusters being assigned to the closest existing cluster-specific models (see Appendix D.3). Building on this potential, we plan to extend FLUX to support dynamic scenarios, enabling adaptive clustering and model updates address evolving client data distributions.
8 Conclusion
In this work, we proposed FL.UX, novel FL. framework that addresses the four most common types of data distribution shifts (P(X), P(Y), P(Y]X), and P(X without requiring any prior knowledge of client data distributions, such as the number of clusters. By leveraging client-side extraction of representative descriptors and an unsupervised clustering approach, FLUX achieves superior performance and robustness across varying levels of heterogeneity and increasing numbers of clients. Unlike existing methods, FLUX enables testing-time association, allowing unseen and unlabeled clients to utilize the most suitable cluster-specific model obtained during training, addressing critical gap in real-world FL applications. Moreover, FLUX incurs minimal computational and communication overhead, comparable to FedAvg, making it scalable and practical for large-scale FL deployments. Our results establish FLUX as a flcxible, scalable, and robust solution for FL, paving the way for more practical and adaptable frameworks in decentralized and privacy-preservinn MI..
Acknowledgments and Disclosure of Funding
This research was funded by the Swiss National Science Foundation, the European Union's 's Hori- zon Europe programme, and the Slovenian Research and Innovation Agency through the projects SmarCCHANGE (No. 101080965), TRUST-ME (No. 20512IL_ 2149211449,, and XAI-PAC (No. 7.00P_21166 21645). PB acknowledges support from the Swiss National Science Foundation Postdoctoral Fellowships IMAGINE (No. 224226).
10

---

References
[1] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication Cimmunicationn-fffcientt Learning of Deep Networks from Decentralized Data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, pages 1273-1282. PMLR, April 2017.
[2] Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloé Kiddon, Jakub Konečný, Stefano Mazzocchi, Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. Towards Federated Learning at Scale: System Design. Proceedings of Machine Learning and Systems, 1:374-388,
April 2019.
[3] Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated Learning: Chal- lenges, Methods, and Future Directions. IEEE Signal Processing Magazine, 37(3):50-60, May 2020. ISSN 1558-0792. doi: 10. 00.0P.20222000099975749999
[4] Dario Fenoglio, Daniel Josifovski, Alessandro Gobbetti, Mattias Formo, Hristjjan Gjoreski, Martin Gjoreski, and Mare Langheinrich. Federated Learning for Privacy-aware Cognitive Workload Estimation. In Proceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia, MUM '23, pages 25-36, New York, NY, USA, December 2023. Association for Computing Machinery. ISBN 979-8-4007-0921-00 doi: 10.1145/3626705. 3627783.
[5] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated Optimization in Heterogeneous Networks. Proceedings of Machine Learning and Systems, 2:429 450, 2020.
[6] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. SCAFFOLD: Stochastic controlled averaging for federated learning. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 Proceedings of Machine Learning Research, pages 5132-5143. PMLR, 13-18 Jul 2020. URL https: htt.htts::/pproeeeee http///pooeeeens..////// mlr press/v119/ karimireddy20a html.
[7] Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurelien Bellet, Mchdi Bennis, Ar- jun Nitin Bhagoji, Kallista Bonawitz, et al. Advances and Open Problems in Federated Learning. Foundations and Trends@ in Machine Learning, 14(1-2):1-210, June 2021. ISSN 1935-8237,
1935-8245. doi: 10.1561/2200000088.
[8] Zili Lu, Heng Pan, Yueyue Dai, Xueming Si, and Yan Zhang. Federated Learning With Non-III Data: A Survey, IEEE Internet of Things Journal, 11(11):1188--9920) June 2024. ISSN 2327-4662. doi: 10.1 10.11099/10T.2224433365
[9] Dario Fenoglio, Gabriele Dominici, Pietro Barbiero, Alberto Tonda, Martin Gjoreski, and Mare Langheinrich. Federated Behavioural Planes: Explaining the Evolution of Client Behaviour in Federated Learning. Advances in Neural Information Processing Systems, 37:1 37:112777-112813, December 2024.
[10] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach. In Advances in Neural Information Processing Systems, volume 33. pages 3557-3568. Curran Associates, Inc., 2020.
[11] Yongxin Guo, Xiaoying Tang, and Tao Lin. FedRC: Tackling Diverse Distribution Shifts Challenge Federated Learning by Robust Clustering, June 2024.
[12] Canh T. Dinh, Nguyen Tran, and Josh Nguyen. Personalized Federated Learning with Moreau Envelopes. In Advances in Neural Information Processing Systems, volume 33, pages 21394- 21405. Curran Associates, Inc., 2020. URL httpp://///
neurips. wwwurpp..c/ppee cc/paper__...p.
ht1ee//paper/222/hnasst http://aaaaeee1111121112aaaaaaa. http:////..../....cccc.ceo/////0 Abstrrctt.b.ttt............aa... atttract.httl. html.
[13] Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi. Adaptive Personal- ized Federated Learningg November 2020. URL. http://arxiv. wwtg/////g//.....//////// 13461. 1rXiv:2003.13461 [cs, stat].

---

[14] Felix Sattler, Klaus-Rober Müller. and Wojeiech Samek. Clustered Federated Learning: Model- Agnostic Distributed Multitask Optimization Under Privacy Constraints. IEEE Transactions on Neural Networks and Learning Systems, 32(8) August 2021. ISSN 2162-2388. doi:
[15] Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An Efficient Framework for Clustered Federated Learning. In Advances in Neural Information Processing Systems, volume 33, pages 19586-19597. Curran Associates, Inc., 2020.
[16] Othmane Marfoq, Giovanni Neglia, Aurélien Bellet, Laetítia Kameni, and Richard Vidal. Federated Multi-Taske Learning under a Mixture of Distributions. In Advances in Neural Information Processing Systems, volume 34, pages 2021.
Curran Associates, Inc.,
[17] Ellango Jothimurugesan, Kevin Hsich, Jianyu Wang, Gauri Joshi, and Phillip B. Gibbons. Federated Learning under Distributed Concept Drift. In Proceedings of The 26th International Conference Artificial Intelligence and Statistics, pages 5834 5853. PMLR, April 2023.
[18] Guodong Long, Ming Xie, Tao Shen, Tianyi Zhou, Xianzhi Wang, and Jing Jiang, Multi- center federated learning: Clients clustering for better personalization. World Wide Web, 26(1): 481-500, January 2023. ISSN 1573-1413 1413. doi: 110077/112800002--000666
[19] Liangze Jiang and Tao Lin. Test-Time Robust Personalization for Federated Learning. The Eleventh International Conference Learning Representations, September 2022.
[20] Wenxuan Bao, Tianxin Wei, Haohan Wang, and Jingrui He. Adaptive Test-Time Personalization for Federated Learning. Advances in Neural Information Processing Systems, 36:77882-77914, December 2023.
[21] Jie Lu, Anjin Liu, Fan Dong, Feng Gu, João Gama, and Guangquan Zhang. Learning under Concept Drift: A Review. IEEE Transactions on Knowledge and Data Engineering, 31(2)) 2346-2363, December 2019. ISSN 1558-2191. doi: 10.1109/TKDE.2018.2288
[22] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, et al. WILDS: A Benchmark of in-the- -n-the-Wild Distribution Shifts. In Proceedingy of the 38th International Conference Machine Learning, pages 5637-5664. PMLR, July 2021.
[23] Wendi Li, Xiao Yang, Weiqing Liu, Yingce Xia, and Jiang Bian. DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptation. Proceedings of the AAAI Conference on Artificial Intelligence, 3694))4092-4100, June 2022. ISSN 2374-3468. doi: 10. (099/aai.v36i4 20327. URL. https://ojs aaai. org/inder///////. php/ AAAI htt///ariileevviiee wwt.e//////.......
[24] Ashraf Tahmasbi, Ellango Jothimurugesan, Srikanta Tirthapura, and Phillip B. Gibbons. Drift- Surf: Stable-State Reactive-Staaie Learning under Concept Drift. In Proceedings of the 38th International Conference on Machine Learning, pages 10054-10064, 10064. PMLR, July 2021. URL
attps://proceediiggs mlr press/ ppess/vi39) htt///v1//////eee http///aaaaabial.. htt.slllliaa.l.ll...m..mmm. html. ISSN: 2640-3498.
[25] Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H. Vincent Poor. Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization. In Advances in Neural Information Processing Systems, volume 33, pages 761 1-723. Curran Associates, Inc., 2020.
[26] Durmus Alp Emre Acar, Yue Zhao, Ramon Matas, Matthew Mattina, Paul Whatmough. and Venkatesh Saligrama. Federated Learning Based on Dynamic Regularization. In International Conference Learning Representations, October 2020.
[27] Cynthia Dwork. Differential privacy. In International colloquium on automata, languages, and programming, pages 1-12. Springer, 2006.
[28] Yann LeCun and Corinna Cortes. The mnist database of handwritten digits. 2005.
12

---

[29] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms, September 2017. URL http://arxiy org/ abs/ 0774777777777....
arXiv: 100.1070077777 [es].
[30] Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images.
[31] Jeremy Irvin, Pranav Rajpurkar. Michael Ko, Yifan Yu, Silviana Ciurea-Ileus, Chris Chute, et al. CheXpert: Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):590-55) July 2019. ISSN 2374-3468. doi: 10. 1el.1669aaiiv3300000 v33101.3301 590. URL https://oje aaai iaiioon org/ index. PhppAAAI/articlee
Number: 01.
[32] Hemanth Venkateswara, Jose Fusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5018-5027, 2017.
[33] Donald Shenaj, Eros Fani, Marco Toldo, Dehora Caldarola, Antonio Tavera, Umberto Michieli, Marco Ciccone, al. Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learning. In JEEE/CVE Winter Conference on Applications of Computer Vision (WACV), pages 444-454, 2023. doi: 10. N0.1109/WAC356688. 0023.0002..
[34] Alessandro Licciardi, Davide Leo, Eros Fani, Barbara Caputo, and Marco Ciccone. Interaction- Aware Gaussian Weighting for Clustered Federated Learning. Forty-Second International Conference Machine Learning, June 2025.
[35] Christopher Briggs, Zhong Fan, and Peter Andras. Federated learning with hierarchical cluster- ing of local updates to improve training on non-IlD data. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1-9, 2020. doi: 10.1
[36] Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. Federated Learning with Matched Averaging, February 2020.
[37] Fei Chen, Mi Luo, Zhenhua Dong, Zhenguo Li, and Xiuqiang He. Federated Meta-Learning with Fast Convergence and Efficient Communication, December 2019.
[38] Yihan Jiang, Jakub Konečný, Keith Rush, and Sreeram Kannan. Improving Federated Learning Personalization via Model Agnostic Meta Learning, January 2023.
[39] Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary. Federated Learning with Personalization Layers, December 2019.
[40] Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting Shared Representations for Personalized Federated Learning. In Proceedings of the 38th International Conference on Machine Learning, pages
PMLR, July 2021.
[41] Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. FedBN: Federated Learning on Non-IID Features via Local Batch Normalization, May 2021.
[42] Othmane Marfoq, Giovanni Neglia, Richard Vidal, and Laetitia Kameni. Personalized Federated Learning through Local Memorization. In Proceedings of the 39th International Conference on Machine Learning, pages 15090-15092. PMLR, June 2022.
[43] Zhuangdi Zhu, Junyuan Hong. and Jiayu Zhou. Data-Free Knowledge Distillation for Heteroge- neous Federated Learning. In Proceedings of the 38th International Conference on Machine Learning, pages 12878-12888 PMLR, July 2021. URL https: http://poocceeeeeeeeeeee/ mlr, .pres/ v v139/zhu21b html. ISSN: 2640-3498.
[44] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach. In Advances in Nearal Information Processing Systems, volume 33, pages 3557-3568. Curran Associates,
Inc., 2020. URL https: http:///roeeeeen.neeeeee neurips. cc/ cttpapprccppppp.....p/. fears_ffleeepepee/eee httpp/ppppepep//aaaa/aee 22.3990reef222ebbbbfra00000000..
Abstract.ht.t html.
13

---

[45] Siddharth Divi, Yi-Shan Lin, Habiba Farrukh, and Z. Berkay Celik. New Metrics to Evaluate the Performance and Faimnsss of Personalized Federated Learning, July 2021. URL http: //arxiv. orgg/bss/110/ 13173. arXiv:2107.13173 [cs].
[46] Li Li, Moming Duan, Duo Liu, Yu Zhang, Ao Ren, Xianzhang Chen, Yujuan Tan, and Chengliang Wang. FedSAE: A Novel Self-Adaptive Federated Learning Framework in Hetero- geneous Systems. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1-10, July 2021. doi: 10. L1000I0NNNN23387222222222 .t..2.2...13778 URL https://ieeexpler ieee. org/oo////oo abbsrract/aoouuutt/S555333
ISSN: 2161-4407.
[47] Xin Yao and Lifeng Sun. Continual Local Training For Better Initialization Of Federated Models. In 2020 IEEE International Conference Image Processing (ICIP), pages 1736-1740, October 2020.
doi: 10.1 100///////1/4 9190068. URL htttts::/iieeexploree ieee. ISSN: 2381-8549.
[48] Chaoyang He, Murali Annavaram, and Salman Avestimehr. Group Knowledge Trans- fer: Federated Learning of Large CNNs at the Edge. In Advances in Neu- ral Information Processing Systems, volume 33, pages 14068-14080. Curran Asso- ciates, Inc., 2020. URL https://prrceedingss wtw.eeeiinnnnn.urrps.........e.e newrips cc/ pccpppprr/222/haah/ a11440001122/1113443000fff hwwprrrreesccca--bbbaaatttteetee Abstract html.
[49] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated Learning with Non-III Data. 2018. doi: 10.48550/arXiv. 1ww.188660082. URL http://arxiv. org/abs/1806. 005882. arXiv: 180.vv::806..... 1806.00582 [cs, stat].
[50] Eunjeong Jeong, Seungeun Oh, Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. Communication-Elfiiccnn On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data, October 2023. URL http://arxiv, org. abs/
1811. 11479. arXiv:1811. 1449 [cs, stat].
[51] Hao Wang. Zakhary Kaplan, Di Niu, and Baochun Li. Optimizing Federated Learning Non-IID Data with Reinforcement Learning. In JEEE INFOCOM 2020 IEEE Conference on Computer Communications, pages 1698-1707, Toronto, ON, Canada, July 2020. IEEE, ISBN 978-1-7281 6--12-0. doi: 10. 1AA.I/FOOOOO44003322001115555010 URL https:// (alocumment99154494444
ieeexplore. ieee.
[52] Moming Duan, Duo Liu, Xianzhang Chen, Renping Liu, Yujuan Tan, and Liang Liang. Self- Balancing Federated Learning With Global Imbalanced Data in Mobile Systems. IEEE Transac- tions on Parallel and Distributed Systems, 32(1):59-71, January 2021. ISSN 1558-2183. doi: 10. 1009/TPDS.2200300000 document/9141455
URL https: https://ieeexploree ieee. woree.or//ee. org.oog///aooo org/abstract/
[53] Qinbin Li, Bingsheng He, and Dawn Song. Model-Contrastive Federated Learning. pages 10713-10722, 2021. URL Li
thecvf hamm//cotett//Ceo http://onteeet////22///tttt /CVPR2221/htm/ _aperrppprrrr html.
[54] Tian Li, Shengyuan Hu, Ahmad Beirami. and Virginia Smith. Ditto: Fair and Robust Federated Learning Through Personalization. In Proceedings of the 38th International Conference on Machine Learning, pages 6357-6368. PMLR, July 2021.
[55] Michael Zhang, Karan Sapra, Sanja Fidler, Serena Yeung, and Jose M. Alvarez. Personalized Federated Learning with First Order Model Optimization. In International Conference on Learning Representations, October 2020.
[56] Rakibul Hasan Rajib, Akil Raihan Iftee, Mir Sazzat Hossain, Sajib Mistry, M Ashraful Amin, and Amin Ahsan Ali. FedCTTA: A Collaborative Approach to Continual Test-Time Time Adaptation in Federated Learning.
[57] Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive Test-Time Adap- tation. In 2022 JEEF/CVF Conference on Computer Vision and Puttern Recognition ( (CVPR), pages 295-305, New Orleans, LA, USA, June 2022. IEEE ISBN 978- 1-66546-946-3 doi: 202.00003....000999.
14

---

[58] Haoyuan Liang, Xinyu Zhang, Shilei Cao, Guowen Li, and Juepeng Zheng. TTA-FedDG: Leveraging Tese-Time Adaptation to Address Federated Domain Generalization. Proceedings of the AAAI Conference on Artificial Intelligence, 19818)118688118888 8658-18666, April 2025. ISSN 2374-3468. doi: 10. 20.1609/aaai.v333 w39.11..3900....5
[59] Jiayu Zhou, Jianhui Chen, and Jieping Ye. Clustered Multi-Tass Learning Via Alternating Structure Optimization. In Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011.
[60] Mahdi Morafah, Saeed Vahidian, Weijia Wang, and Bill Lin. FLIS: Clustered Federated Learning Via Inference Similarity for Non-IID Data Distribution. IEEE Open Journal of the Computer Society, 4: 109-120, 2023. ISSN 2644-1 1268. doi: 10.1 10.1109/OJCS.. 202.3222203..
[61] Luxin Cai, Naiyue Chen, Yuanzhouhan Cao, Jiahuan He, and Yidong Li. FedCE:
Personalized Federated Learning Method based on Clustering Ensembles. In Proceedings of the 31st ACM International Conference on Multimedia, MM '23, pages 1625-1633, New York, NY, USA. October 2023. Association for Computing Machinery. ISBN 9798400701085. doi: 10.1 773.38817733361217777
[62] Honglan Huang, Wei Shi, Yanghe Feng, Chaoyue Niu, Guangquan Cheng. Jincai Huang, and Zhong Liu. Active Client Selection for Clustered Federated Learning. IEEE Transactions on Neural Networks and Learning Systems, (TT1)::66444116443 November 2024. ISSN 2162-2388, doi: 10.1 111.1//NNNS.0023.32222222
[63] Chengxi Li, Gang Li, and Pramod K. Varshney. Federated Learning With Soft Clustering, IEEE Internet of Things Journal, 9(10)7773-7782, May 2022. ISSN 2327-4662. doi: 10.1109/JIO. 2021.3113922
[64] Biyao Gong, Tianzhang Xing, Zhidan Liu, Wei Xi, and Xiaojiang Chen. Adaptive Client Clustering for Efficient Federated Learning Over Non-IID and Imbalanced Data. IEEE Transactions on Big Data, (0(6):1051-100 December 2024. ISSN 2332-7790.
doi:
0..100//BDAAT.2202..31160
[65] Moming Duan, Duo Liu, Xinyuan Ji, Yu Wu, Liang Liang, Xianzhang Chen, Yujuan Tan, and Ao Ren. Flexible Clustered Federated Learning for Client-Level Data Distribution Shift. IEEE Transactions on Parallel and Distributed Systems, (3(11):2661-261)) 2022. doi: 10. 109/
TPDS.2021.3134D2
[66] Joel Wolfrath, Nikhil Sreekumar, Dhruv Kumar, Yuanli Wang, and Abhishek Chandra. HACCS: Heterogeneity-AWee Aware Clustered Client Selection for Accelerated Federated Learning. In JEEE International Parallel and Distributed Processing Symposium (IPDPS), pages 985-995, May
2022. doi: 10. 1 10.100///PPS53211.202 2002.000000
[67] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized Federated Learning: A Meta-Leaming Approach, October 2020.
[68] Enmao Diao, Jie Ding, and Vahid Tarokh. HeteroFL.: Computation and Communication Efficient Federated Learning for Heterogeneous Clients. In International Conference Learning Representations, October 2020.
[69] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-basse learning applied to doc- ument recognition. Proceedings of the IEEE, 86(11):2278-233) November 1998. ISSN 1558-2256. doi: document/726791. Conference Name: Proceedings of the IEEE.
URL https: s/itee://iieeeplorree ieee. www..orggbbaba.eeee abbstract/////////
[70] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pages 770-778, Las Vegas, NV, USA, June 2016. IEEE. ISBN 977-1-4673-8881-11 doi: 10.1109/CVPR200
URL http::/ieeexpporeeeee ieee. htt.:/rrw...e.eeeeeeeeeeeeee/e/e org//occmen//780455//
[71] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, et al. PyTorch: An Imperative Style, High-Peeformance Deep Learning Library. Advances in Neural Information Processing Systems, 32, 2019.
15

---

[72] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12:2825-2888 28300, 2011.
[73] Daniel J. Beutel, Taner Topal. Akhil Mathur, Xinchi Qiu, Javier Gernandez-Marqueee Yan Gao, Lorenzo Sani, Kwing Hei Li, Titouan Parcollet, Pedro Porto Buarque de Gusmão, and Nicholas D. Lane. Flower: A Friendly Federated Learning Research Framework, March 2022.
[74] J. D. Hunter. Matplotlib: A 2D graphics environment. Computing in Science & Engineering, 9 (3):90-95, 2007. doi: 10.100//CS222007.5555
[75] Michael L. Waskom. seaborn: statistical data visualization. Journal of Open Source Software 6(60):3021, 2021.
doi: 10.21105/joss.0000 03021. URL attps://doi. www.oo..//....o...ooooooo... 21105///...11///ss/.
03021.
[76] Wes McKinney. Data Structures for Statistical Computing in Python. In Stéfan van der Walt and Jarrod Millman, editors, Proceedings of the 9th Python in Science Conference, pages 56 61, 2010. doi: 10.30090Maioon---b1199200000
[77] Cynthia Dwork, Frank McSherry. Kobbi Nissim, and Adam Smith. Calibrating Noise to Sensitivity in Private Data Analysis. In Shai Halevi and Tal Rabin, editors, Theory of Cryp- tography, pages 265-284, Berlin, Heidelberg, 2006. Springer. ISBN 978-3-540-32732-5 doi: 10.1007/11681878 14.
[78] Cédric Villani et Optimal transport: old and new, volume 338 of Grundlehren der Mathema- tischen Wissenschaften. Springer, 2008.
[79] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership Inference Attacks Against Machine Learning Models. 2017 IEEÉ Symposium on Security and Privacy (SP), pages 3-18, May 2017. doi: 10. 109/SP.201...411..
[80] Oualid Zari, Chuan Xu, and Giovanni Neglia. Efficient passive membership inference attack in federated learning, October 2021.
[81] Jiacheng Li, Ninghui Li, and Bruno Ribeiro. Effective passive membership inference attacks in federated learning against overparameterized models. In The Eleventh International Conference on Learning Representations, September 2022.
[82] Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS '17, pages 603-616 New York, NY, USA. October 2017. Association for Computing Machinery. ISBN 978-1-4503 4946-8. doi: 10.1145/3133956.31344
[83] Ligeng Zhu, Zhijian Liu, and Song Han. Deep Leakage from Gradients. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[84] Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M. Alvarez, Jan Kautz, and Pavlo Molchanov. See Through Gradients: Image Batch Recovery via GradInversion. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16337-16346. 2021.
16

---

APPENDIX CONTENTS
A Related Approaches Distribution Shift
18
A.1 Clustered Federated Learning
18
A.2 Personalized Federated Learning.
19
A.3 Test-time Adaptive Federated Learning.
20
B Reproducibility & Implementation Overview
20
B.1 Datasets and Non-IID Generation Protecol
20
Models pprammrr Settings
................................
21
B.3 Code, Licenses and Hardware
22
B.4 Algorithm Pseudo-code
22
C Descriptor Extractor Details and Theoretical Guarantees
22
C.I. Descriptor Extractor
22
C.2 Privacy Implications of Distribution Descriptors in FLUX
FL..............................
26
Advantages Latent Space Descriptors Neural Networks
27
C.4 Identification Distribution Shifts
27
D Clustering Mechanics
28
D.1 Dynamic Clustering Initiation
28
D.2 Unsupervised Clustering Setup
29
D.3
Empirical Illustration of Clustering
30
E
Baselines & Comparative Complexity
31
E.1 Overview Baseline Algorithms
31
E.2
Comparative Analysis of lleterogeneity and Computational Costs
32
E.3
Comparison with Descriptor-based Methods
34
F
Additional Experiment Results
35
F.1
Scaling Data Heterogeneity Types and Levels Scaling the Number Clients
.o..............................
35
F22
................................
36
Robustness Under Mixed Distribution Shifts
F.4 Results on Real-world Datasets
52
F.5
Robustness to Partial Participation
54
F6
Ablation Studies
54
17

---

A Related Approaches to Distribution Shift
A wide range of methods has been proposed address data heterogeneity in FL., which can be grouped into three main categories-clusten FL (CFL), personalized FL (PFL), and test-time adaptive FL (TTA-FL). Below, we summarize each category's 's key principles and limitations.
A.1 Clustered Federated Learning
CFL assumes the presence of M distinct data distributions within the federation and aims to learn one model per distribution. Existing CFI. methods can be categorized by their clustering strateyy-eithe as hard or soft clustering- by the underlying clustering principle, namely metric-based or parameter-based (gradient-based) approaches.
A.1.1 Clustering Strategy: Hard vs. Soft CFL
In Equation 2. we introduced general definition of CFL that unifies hard- and soft CFL under single framework. This formulation follows standard mixture-model derivations: soft-CFL defines client assignments c(*) as probability vectors, while hard-CFL is the special case where c( reduces one-hot vector. Hence, Equation 2 can be seen surrogate objective derived from the soft-CFL likelihood via the standard Jensen lower bound, subsuming both variants under a single expression. This appendix explicitly presents the optimization problems for hard-CFL and soft-CFL, as is standard in the literature, to provide greater clarity on their specific formulations.
Hard-CFi assigns each client & K. to exactly one cluster based predefined criteria similar- ity measures. Therefore, each cluster Cm a subset of K., i.e., Cm S K. for all m E (1....,M4 M). Additionally, the clusters are disjoint and collectively exhaustive, satisfying U Cm K. and Cn n G2 = 0 for all * n. This means that each cluster Gm includes a distinct subset of clients with similar data distributions, allowing the hard-CFL. framework to train specialized models tailored to each cluster's 's specific data distribution. Formally, hard-CFL seeks to jointly optimize the model parameters and the cluster assignments in Equation 10. This formulation ensures that each -luster-specific model 0r,, minimizes the risk for all clients within cluster Cyss and generally suppose to know the number of clusters M.
M
10th M
mm11 C* arg
maX (6... mas
log P(y] Om),
(10)
(a (00))
Soft-CFL allows clients K. to belong multiple clusters with certain probabilities or weights, accommodating scenarios where client data may exhibit overlapping distributions that can be effectively modeled by combination of specific models. Therefore, each client is associated
(A) with a weight vector N (k) = {-] ..... ग is ). where (k) 2 0 and >m (N) = 1. This enables each client contribute multiple cluster-specifie models based on weights. Mathematically,
soft-CFL seeks to jointly optimize the cluster-specific model parameters { and the
assignment weights {T(()) Jum as follows:
K
1M {nr (k),a) ) arg max
Σ
Tm log P(ylæ; 0m),
pk=1
{0.. m 1 (
F8E1
(c.))((((()gg0))
(11)
This formulation allows each client be influenced by multiple cluster-speeific models, thereby enhancing the flexibility of cluster assignment. However, optimizing both cluster-specific models and assignment weights (+) simultaneously poses additional computational and convergence challenges, requiring alternative optimization techniques [59]. Furthermore, soft-CFL is generally not designed to handle unseen clients effectively. determining assignment weights (4) requires access to the training data of these clients [11. 16].
A.1.2 Clustering Principles: Metric-Based vs. Parameter-Basse CFL
CFL can be further classified into two categories based on their clustering principles: metric-based CFL and parameter-buse CFL. Table provides a categorization of some existing CFL. frameworks according to these principles.
18

---

CLUSTERING PRINCIPLE
FRAMEWORK
METRIC-BASER
IFCA [15]
FEDEM [16]
FEDDRIFT [17]
FLIS [60]
FEDCE [61]
ACFL [62]
FLSC
FEDGWC [34]
PARAMETER BASED
CFL [14]
PRSEM [18]
FEDRC [11]
FL+HC [35]
AUTOCFL [64]
FLeXCFL [65]
Table 3: Summary of CFL frameworks categorized by clustering principles.
Metric-Based CFI These algorithms assign cluster identities based on model losses or inference- phase accuracy. Typically, the server broadcasts M models corresponding each client selects the model that minimizes the empirical risk on its local aatta-sseetiaalle identifying the closest model its cluster. Alternatively, client-side loss or accuracy values can be directly utilized by the server for clustering. However, this approach relies on highly compressed information, which can lead to inaccuracies. For instance, two clients with distinet data distributions might achieve the same accuracy by making errors on different samples, causing them to be grouped in the same cluster.
M clusters, and
Parameter-Bal -assed CFL. These algorithms perform clustering based on the parameters or gradient updates of clients' models. When clients possess non-IID datasets, local training epochs result in diverging model parameter updates, which can be utilized for cluster identification. Two common approaches for parameter- baseed CFL are: the server broadcasts M models, and each client selects the one closest to its updated model, and ii) the server directly clusters clients' updated model parameters or gradients. However, relying solely on distance metrics for clustering may also lead to inaccuracies, as such metrics provide overly condensed information, making it challenging to differentiate intrinsically non-Ill) clients.
FLUX explicitly employs descriptor extractor to capture statistical characteristics of client data. does not fall neatly into either category, as it clusters based data-distribution descriptors rather than model parameters or compressed metrics. The closest related approach is HACCS [66]. which also introduces descriptor-like summaries through label and pixel-value histograms. However, HACCS lacks semantic and structural awareness (pixel statistics may not reflect task-relevant features), requires labels inference (preventing test-time adaptation). and produces descriptors whose size grows with the number of labels and bins. Moreover, its histograms are human-interpretable raising stronger privacy concerns (e.g., revealing label distributions), and requiring strong differential privacy. In contrast, F1.UX leverages compact latent representations aligned with the classification objective, which are non-invertible and task-relevant, enabling robust clustering across all four types of distribution shift, and generalization to unseen clients. Numerical results comparing FL.XX with HACCS are provided in Section E.3.
A.2 Personalized Federated Learning.
PFL addresses client heterogeneity by shifting from a global objective to a client-specific optimization framework, where each client & learns tailored model optimized for its local data distribution. Instead of enforcing a single shared model across all participants, PFL explicitly accounts for non-IID data by enabling model personalization. Existing methods can be broadly classified into three categories:
Fine-tuning. These approaches begin with a shared global model that is subsequently adapted locally using additional gradient steps or meta- meta-learning techniques simplify personalization [37, 67, 38]. While conceptually simple, they often require careful tuning of hyperparameters and sufficient local data avoid overfitting.
Model decoupling. These methods separate the model into shared (global) and client-specific components, such training common backbone alongside global and personalized heads [39,
19

---

13, 40, 19, 42]. Other variants adapt only batch-norm norm statistics [41] or allow for heterogeneous encoder architectures [68]. These approaches enhance model expressiveness but incur higher on-device memory and computational overhead.
Regularization-baseed These approaches personalize models by adding regularization term that encourages proximity between each client's 's local model ((4) and the global model 9. A typical
formulation augments the local objective with penalty term: min ( (x)))) + ||o(()
0|| where À controls the trade-off between personalization and global consisteney [67, 54]. While these methods offer smooth personalization, they often require tuning client-specific hyperparameters and solving nested or bi-level optimization problems.
Although PFL methods are effective when sufficient labeled data is available each client, their per- formance degrades in low-data regimes. Moreover, they typically introduce additional computational and memory overhead on the client side, ood-being inherently supervised-arg are not applicable to unseen, unlabeled clients that appear only at test time.
A.3 Test-time Adaptive Federated Learning.
TTA-FL targets post-deployne distribution shifts by adapting pre-trained global model each client's unlabeled test data. Most approaches optimize unsupervixed objectives- as entropy minimization [19, 20, 56] or self-supervised contrastive losses [57, directly on the test set. However, in the absence of labels, the resulting optimization landscape is often unstable: entropy minimization can lead to overconfident mispredictions under concept shift, while contrastive losses may collapse small or imbalanced test batches-a common constraint in FL deployment. To improve stability, recent methods constrain adaptation to small set of parameters (c.g.. batch-norm statistics or interpolation weights between global and personalized heads [19]). Nonetheless, these models still require labeled data during training to learn the personalized components, limiting their applicability truly unseen and unlabeled clients test time.
B Reproducibility & &Immplemetatton Overview
B.1 Datasets and Non-IID Generation Protocol
For MNIST, FMNIST, CIFAR-10, and CIFAR-100 100, we generate the four most common types of distribution shift using ANDA $ (A Non-IID Data generator supporting Any kind), toolkit designed to create reproducible non-IID datasets for FL experimentation. It supports datasets MNIST, EMNIST, FMNIST, CIFAR-10, and CIFAR-100 100, and facilitates five types of data distribution shifts: feature distribution shift, label distribution shift, X( concept shift, P(X concept shift, and quantity shift.
ANDA.
Figures 6 and 7 illustrate examples of MNIST and CIFAR-I0 datasets under four distinct types of data distribution shifts generated using ANDA:
In (a), the two clients experience different feature distribution shifts. Each image undergoes one of three color transformations (blue, green, or red) and one of four rotations (0s, 90", 180,, or 270"). with distinct distributions applied each client. For instance, the first client has a higher proportion of blue images, while the second client has more red images. In (b), the clients experience label distribution shifts. Each client receives data from only three classes. For example, in the MNIST dataset, the first client has images of digits 0, 3, and 6, while the second client has images of digits 1, 2, and 4.
In (c), the clients exhibit P(Y XX )cocnccppt shifts. Given identical feature distributions (e.g.. image pixels), labels differ between clients. For instance, in the MNIST dataset, the first client labels the digit '1' as 'I' and '2' as '2', whereas the second client labels '1' as '2' and '2' as '1'.
In (d), the clients demonstrate P(X (xX(Y)-concpp. shifts. For the same label, different features are applied. For example, in the MNIST dataset, the first client applies a red hue to images labeled '2', while the second client applies blue hue to the same label.
In subsequent sections, we detail the specific dataset generation settings employed with ANDA.
20

---

8
6
O
3
3
9
0
4
3
3
3
3
2
6
N
O
O
0
4
2
5
30
C
6
6
0
6
3
2
3
b
6
O
6
6
(b)
0
6
/
8
2
6
6
/ 0 2 4
O
0
(
6
/
8
2
6
O
/
2
6
6
3
9 9
3
/
X
2
6
6
3
O
3
7
5
4 O
7
5
9
7
4
9 2
2
5
9
7
2
e
8
9
7
3
3
9 a
9
3
b
8
3 3
9
2
Figure 6: MNIST datasets with four different types of data distribution shifts generated by ANDA.
Figure 7: CIFAR-10 datasets with four types of data distribution shifts generated by ANDA.
CheXpert. For the real-world CheXpert dataset, we retain the original data without applying any image augmentations label modifications preserve the correctness and clinical integrity of the data. To simulate varying levels of client heterogeneity, we partition the dataset into multiple distributions using three available metadata attributes: ViewPosition, Age, and Sex. Specifically, construct the following non-IID configurations:
Low heterogeneity:
distributions based on ViewPosition (Frontal vs. Lateral).
Medium heterogeneity: 4 distributions based on ViewPosition and Age (<50 vs. 250).
High heterogeneity: 8 distributions based ViewPosition, Age, and Sex (Male vs. Female).
For the real-world dataset Office-Home, we do not apply any image augmentation or label modification; instead, we directly the dataset in its original form. For experiments, restrict the dataset to images with only 20 classes. To model different levels of data heterogeneity. we leverage the four inherent domains of the dataset: Art, Clipart, Product, and shift across these categories naturally induces heterogeneity in the data distribution.
Office-Home.
The domain
B.2 Models and Hyper- Srpper-paramete Settings
adopt a 5-fold cross-validation strategy to evaluate model performance, using fixed random seeds (42, 43, 44, 45, and 46) to ensure reproducibility. For the MNIST, FMNIST, and CIFAR-10
21

---

datasets, we use LeNet-5 [69] as the base model; for CIFAR-100, CheXpert, and use ResNet-9 [70]. A batch size of 64 is used for both training and evaluation. Each client reserves 20% of its local data for validation. The FL. process runs for 10 communication rounds on MNIST, FMNIST, and CIFAR-IO, for rounds on CIFAR-I00, 20 rounds on CheXpert, 40 rounds on Office-Home, with each client performing 2 local training epochs per round. The learning rate is set to 0.005 with a momentum of 0.9. All models are trained using cross-entropy loss, except on CheXpert, where a binary cross-entropy loss is used due to the multi-label classification setting.
we
B.3 Code, Licenses and Hardware
Our experiments were implemented using Python 3.12 and open-source libraries including PyTorch 2.4 [71] (BSD license), Scikit-learn 1.5 [72] (BSD license), and Flower 1.11 [73] (Apache License). For visualization, we utilized Matplotlib 3.9 [74] (BSD license) and Seaborn 0.13 [75] (BSD license), while data processing was performed using Pandas 2.2 [76] (BSD license). The datasets used in our experiments- -MNIST (GNU license), FMNIST (MIT license), CIFAR-IO. 10, CIFAR-I00, CheXpert, and Office-Home are freely available online. To ensure reproducibility, our code, along with detailed instructions for reproducing the experiments, publicly accessible on GitHub under the MIT license. We used publicly available codes for our baselines (except FedAvg).
All experiments were conducted on workstation equipped with four NVIDIA RTX A6000 GPUs (48 GB each), two AMD EPYC 7513 32-Core processors, and 512 GB of RAM.
B.4 Algorithm Pseudo-code
We provide the pseudo-code for both the training phase (Algorithm 1) and inference phase (Algorithm 2) of our proposed implementation of FLUX. These include steps for extracting representative descriptors of client data distributions and performing unsupervised clustering.
C Descriptor Extractor Details and Theoretical Guarantees
C.1 Descriptor Extractor
This section presents the implementation of the descriptor extractor ø, which enables each client to encode its local data distribution in a compact and privacy-preserving way. The resulting descriptor d(*) supports consistent similarity comparisons across clients while satisfying all requirements in Definition 4.2.1. We outline the implementation in four federated steps:
C.1.1 Implementation Details.
Let the final hidden layer of the global model produce, for client k. matrix of latent representations
s(k) E < x2 The descriptor extractor maps these latents to a descriptor (kk) E R through the
following steps:
SI Global alignment (one shot, no raw data). Each client computes the element-wise minimum and maximum of its latents and sends only these two z-dimensional vectors (i.e., 2z floats) to the server. The server aggregates them by coordinate-wise min and max to obtain global bounds [m_,m*] which are broadcast to all clients along with the model weights. This step ensures consistent alignment without transmitting raw data, labels, or gradients.
$2 Shared PCA on synthetic reference points. Using a shared random seed, all clients generate 200 synthetic points uniformly sampled from [m .m+] and independently fit PCA map R= + TR. (with / = 10) to this synthetic set. Because both the sampling and data are identical, all clients derive the same linear projector g, ensuring that Euclidean geometry in the reduced space is aligned across the federation.
S3 Moment computation. Each client projects its latent matrix using the shared PCA: ((k) ((s(k)) E (tx, of 2(1), approximating the marginal distribution P(X), and (ii) class-conditional statistics [(1l)) (k) - "())) approximating the conditional distribution P(Y X).
xt, It then computes: (i) global statistics (n4 (k) I(k), ), the mean and covariance
22

---

Algorithm 1: FLUX Framework - Training Phase
Input: Model f(0), client set K. [1,...,KK } rounds deseriptor extractor ((3)), clustering method A))
Output: Cluster-specific models {0m 1M=1- cluster centroids {ymm b
Descriptor Extraction;
(X)):
// contains clients for initial clustering
for each client & A in parallel do g(k) i
d f- ob (c())) ;()
1/ Extract descriptor
Send d(k), g(h), and ((h) to server; Unsupervised Clustering:
C C (ddd(((a)ae )EEAi)) do
with C= (o c0 E {0,1)*
for m
10 1m + Eied (
11 You Am
1 ExEA Cm (
// Cluster centroids
12 Clustered Federated Learning:
13
for per==
1 to R8 do for m to M do
14
Sm - ー (8)
15
16
0m 5mm -
(ki (8) g(k);
// Weighted aggregation
17
A + RandomClie RandomClientSooocctee (ot)occcoots)))
// New participants each round
for each client & E A without (k)) in parallel do ((k) 0":
18
19
( + -0((00) (() Send d(aa) 800,, and (0()) to server; for each client k A without (( in parallel do (k) arg minm==1..... x6 ((dth)) 7/m
// Descriptor for cluster assignment
21
22
// Model/cluster assignment
for each client & with (k) Cm do
24
25
L Send 6m to client k;
26
for each client & A in parallel do Receive 0mi
27
((h) -0mi
g(k) + Locallrrain ((()) (kk) ,(()) v(t:;
29
// Local update
Send g(k) and (b) to server,
30
$4 Differential privacy (optional plug-in). To enhance privacy guarantees, each statistic gi in the descriptor be independently perturbed using the Laplace mechanism (6 0) [77], detailed in Appendix C.2:
T/. Laplace(O,b:) by),
b; = A1,//,
where AI,: denotes the e ei-sniiitv of the i-th statistic. For means and standard deviations, we conservatively bound A < (Ragge(us//ls) The resulting descriptor is
(k)) .gl " + 7 7~ diagg(,,,..i..,,,....... ba)).
C.1.2 Distribution Fidelity (R1).
To satisfy the condition in Equation 4, we design the descriptor extractor so that Euclidean distances in the descriptor space RR4 closely approximate the 2- 2-Wasserstein distance W2 between client data distributions. This is achieved by globally aligning all clients using consistent, privacy-preservin PCA, and constructing each descriptor from the first two moments of: (i) the marginal latent
distribution P(X). and (ii) the class-conditional latent distributions (P(X Y =u]
23

---

Algorithm 2: FLUX Framework - Inference Phase
Input: set test clients = {1...., Q) cluster-specific models {0m )M cluster centroids (Tm m==1
descriptor extractor ((5 1)), distance function x((--)-
Output: Predictions [
// Descriptor extraction
for each client q
parallel do
d'(n) + ((x(*) 0; 0;
// Extract descriptor
Send d'(9) to server;
// Unsupervised model assignment for each client q E @ in parallel do c=(x) <- arg mittm=l,....M 1s (()(()) 7m))
// Model assignment
Send (2)))) client q;
// Local prediction for each client g E Q in parallel do gl [2 0
// Inference
We base our analysis the following conditions:
(A1) (Gaussian latent approximation) Each latent distribution can be approximated by a Gaussian, i.e. P. ss ((s, E.). This assumption is commonly adopted for deep feature representations. (A2) (Spectral bounds) There exist constants 0 < Amin S Amax < oc such that all covariance eigenvalues remain in the interval [Amin: Amax]. practice, this is enforced to Step $1 of the extractor, which restricts latent values shared bounding box mm+].
After applying the shared PCA. covariances become nearly diagonal. (A3) This allows to treat them as commuting matrices, which yields exact identities; otherwise. standard operator inequalities can be applied.
For clarity, we present the following proposition the marginal setting; the class-conditional version follows by applying it to each class component.
Proposition C.1 (Cipschit--equivalence to W2 for marginals). Consider the distance between two client descriptors defined as
A 11 - 12l1 + ||= E2
(12)
Under assumptions (A3), there exist constants
min(1, (2VAmax)-1 },
C+ max(1, (2vAmin)
such that the following inequality holds:
2 A S WH(N EI).N( Ex) S 2.
Hence, within the admissible covariance the squared 2- 2-Wasserstein distance and the descriptor distance are equivalent up to constant factors.
Proof. To streamline notation, let denote each client's marginal descriptor by ((k) [pks vec(Ex)]] The squared Euclidean distance between two such descriptors is then
(| - 2 12 || #2l1 + ||voe(E; - Σz)11
Since for any matrix A we have (A)e(A))) = || taking A = E, - Eg yields Equation 12:
[1] - 2 = ||ا| 22 + | -2
For Gaussian distributions, the squared 2- 2asssersteinn distance has the closed form W3 = [4 122122+ + (21,,, E2), where (²,,,,, E2) = Tr Σ, + Σn "=/////,,////,/// is the squared Bures distance [78]. The mean terms in (12) and in W3 coincide, so the comparison reduces to relating B(E,, M2) with [[E; Σ211F.
24

---

Non-IID Level
Max
Min
Mean
Std
Non-IID Level
Max
Min
Mean
Std
0.194
28816
0.00017
0.522
5
1.088
0,159
0.479
0.113
3,464
0.00135
1.744
0,551
1.016
0.136
0.458
0.108
3.429
0.00441
1.734
0.590
(a) MNIST
(b) CIFAR-10
Table 4: Absolute error & between descriptor distance (2- 3-Wasserstein) and the true inter-client distance under three non-IID levels for MNIST and CIFAR-I0 under P(Y (X) concept shift.
Assumption (A3) ensures that the covariance matrices commute (Σ,E² ΣgΣn). e.g., they are diagonal after the shared PCA. In this setting,
///////////////////
(2//3 12211 2
Substituting into the Bures expression gives
(M,, 2n) = Tr()))
- 2
= -
But the right-hand side coincides with the expansion of the squared Frobenius norm
||_|// / = Tr(En) Tr /
so we conclude that
(E = || 12/2 1
Next, applying entrywise the mean value theorem for f(x) v on Amin Amax], yield the inequality
Eglls F s BEE,, Ex) S 2 E Σ2llF. |E
2 Amax
Finally, let = |1 U22 1 and b = || -2l p.o and write & E [Rimiu kimax) with kmin 1/(4)max) and kinax = 1/(4Amiu). Then the elementary inequality
main[1,k](c +b) b) I a + kb S max{1, + +)
implies that c²
W < 2 with the constants c_ and c. as stated.
Empirical validation. We empirically validate this guarantee using 44,850 client-round pairs from MNIST and CIFAR-10 under three levels of non-llD concept shift affecting P(Y X) (see Appendix B.1 for protocol details). Table 4 reports the absolute deviation (k)
[|]d(k) d(*>)||2
P(z(k)), ( (ka))) On MNIST, the worst-case error is below 1.1, with mean 0.49 + 0.11. On CIFAR-10, 10, the worst-case reaches 3.5, with mcan of 1.7 0.5. These results confirm that the deseriptor extractor satisfies Equation 4 across heterogeneous distribution shifts, while preserving client privacy.
C.1.3 Label Agnosticism (R2)
In real-world deployments, test-time clients rarely possess reliable labels. To ensure applicability in this setting, Requirement (R2) mandates that each descriptor include sub-vector computable using features only. Our implementation naturally satisfies this through the structure of the moments computed in Step S3.
Training phase, Each client computes both the marginal moments (μ >> and class-conditional moments {pr
(k) (14) ) resulting in descriptor that splits follows:
((k)
(k)
(K)
I (R)
d E ER
(1()) CRL-P
where the sub-vector (k) encodes information from features only and serves as the label-free component.
25

---

Test phase (labels unavailable). At test time, clients repeat Steps SI-S2 and execute only the label-free portion of Step S3, computing (ua, >> from features alone. Since P(Y X) cannot be estimated without labels, no class-conditional moments are computed. The resulting test-time descriptor is:
((())
و, (zt 0) E IR,,
which satisfies the requirement for label-agnostic descriptor construction. When applying Step S4, the same coordinate- wise Laplace mechanism ensures that d((h) enjoys the same (e, (e. guarantee as the full descriptor.
Since the PCA projection shared across clients (Step 52) and noise calibration in Step S4 data- independent, Euclidean distances between label-free descriptors-iie. ||d"(ku) ) dd(k2) (|r6ka/||2-eee le-nmaain a valid proxy for the marginal Wasserstein distance between their respective P(X) distributions. This enables our extractor match unseen, unlabeled clients at test time to the most similar training distributions, thereby fully satisfying Requirement (R2).
C.1.4 Compactness (R3)
Requirement (R3) stipulates that descriptor extraction should incur only marginal computational and communication overhead relative standard FL. Our implementation satisfies this both fronts.
Computation. The only non-trivial operation is computing rank-l PCA SCA 200 synthetic latent vectors of dimension z (Step S2). Using a standard SVD solver, the cost is AminnsPCAz2 (sPCA)2z)). In practice, PCA, so the second term dominates, yielding roughly 4 10 floating-point operations. For typical latent sizes (a results in at most 8.2 x 10 FLOPs-negligibe compared to the cost of one local training epoch. For comparison, even a single forward pass with our smallest MNIST model exceeds 6.5 x 10 FLOPs, while training the largest model on CIFAR- 100 requires over 6 x 10"l FLOPs per epoch. Moment computation (Step S3) and noise addition (Step S4) are linear in descriptor size and thus negligible.
2048). this
Communication. Each descriptor transmits d 2(U + 1)l floats, corresponding to the mean and (diagonal) covariance for the marginal distribution and each of the U classes. With 1 = 10, this amounts to d = 220 floats on MNIST (U = 10) and d 2020 on CIFAR- 100 (U = 100). In contrast, the corresponding model update sizes range from 62,006 parameters (MNIST) to 6,775.140 (CIFAR-100), yielding descriptor-to-modie ratio at most d/p S 3.5 x 10-3 This remains well below the threshold of 10-2 imposed by the compactness requirement.
C.2 Privacy Implications of Distribution Descriptors in FLUX
Because FLUX transmits client-side descriptors {d(*)) JAEK C which, by design, summarize each client's data distribution (Requirement (R1))aan adversary could. in principle, combine them with model updates to mount stronger reconstruction or membership-infeeeenc attacks [79-84]. Two properties mitigate this risk:
Many-to-one mapping. As defined Section 4.2.1, the extractor Φ: (** x(*+u) R performs severe dimensionality reduction (s (z + u) >> L) and aggregates client-level statistics. So from an information- -eemmation-theoretic perspective, infinitely many distinct datasets (zx(*) (/))) can yield the same descriptor d Consequently, deseriptors are non- -oveervttti.le.
2. Differential-privaer wrapper: We can endow each descriptor with (e, 8) diffferrrlt privacy (DP) guarantee at the sample-level [27] by adding calibrated noise on the client-side. This ensure that the impact of any single sample remains bounded by the privacy budget c- without negatively affecting performance. Precisely, a mechanism A satisfies (e, 8)-DP if, for any two neighbouring datasets D and D' differing by at most one entry, and for any measurable subset of outputs S, the following holds:
Pr(M(D) S S e Pr(M(D') $] +8
(13)
where e >> is the privacy budget controlling the allowable difference between the probabilities, and 8 > 0 quantifies the probability of violating the bound.
26

---

Differential privacy integration. In our implementation, we perturb each coordinate of d(x) with independent Laplace noise (6 0) [77]:
ud Laplacd(0,b) 0,,6:)
b, A1,//E
7
where 1 is the fi-sensitivity of statistic 9s. For a mean or standard coordinate we conservatively bound A1, S
Range(a.), The released descriptor is therefore
2(())
(k)) - ga I" +7. ~ Laplace (0, diag
For example, setting a common 10.0; with typical client sizes u(k) > 300, the added variance 2(A1,/E)² I 2.2 10-5 is negligible compared natural data variability, so inter-descriptor distances remain reliable without affecting FLUX performance. Combined with the many-to-one
nature of Φy : R. x(*+u) where infinitely many distinct datasets map to the same descriptor-this DP mechanism bounds any additional leakage to an g-limited factor beyond what is already exposed by model parameters, thereby fully satisfying privacy guarantees.
Known Association
Test Phase
0.01
90.06 # 1.80
57.46 13.26
0.1
92.19 2.11
74.48 # 12.25
91.80 + 2.49
93.80 # 1.42
10
92.49 2.17
93.94 1.40
No DP
92.86 # 2.02
94.36 1.33
Table 5: Effect of Differential Privacy FLUX based on varying values. Results are presented with Known Association and Test Phase conditions, showing accuracy as mean standard deviation.
Experimental validation. As shown in Table 5, we evaluated FLUX under varying privacy budgets 6 to assess the trade-off between privacy and performance. The results demonstrate that FLUX maintains robust performance across commonly used privacy budgets (1 S S 10), achieving comparable accuracy to the non-DP version. Only under extremely stringent privacy settings (e.g., 0.01 and 0.1), which uncommon in practical applications, does FLUX exhibit a noticeable decline in performance, particularly in the test phase. These results confirm that the proposed DP integration provides rigorous privacy guarantees while preserving the effectiveness of FLUX, making suitable for deployment in scenarios requiring strict regulatory compliance.
C.3 Ad Advantages of Latent Space Descriptors in Neural Networks
We decided to extract data descriptors from the latent space of the trained model on the same dataset leverage the inherent benefits of this representation. The latent space offers reduced-dimennsiona representation, encapsulating only the most relevant features while eliminating redundant information. This dimensionality reduction enhances computational efficiency and facilitates downstream tasks without sacrificing the descriptive power of the features.
Furthermore, descriptors from the latent space reflect higher-level feature representations learned by the model. Neural networks are designed to prioritize task-relevant patterns, abstracting away low-level details in favor of semantically rich and domain-specinf features. This makes latent space descriptors more informative and better aligned with the underlying classification task. Additionally, these representations are inherently resilient to noise and input variability, as the training process filters out irrelevant perturbations. This robustness ensures consistent and reliable descriptors, even under transformations or distortions the input data. By aligning with the model's learned domain- specific knowledge, latent space descriptors also provide a task-informed perspective that enhances their interpretability and relevance. These advantages collectively establish the latent space superior choice for extracting meaningful and efficient data descriptors, especially in scenarios where robustness, efficiency, and task specific alignment are critical.
C.4 Identification of Distribution Shifts
Figure offers an intuitive view of the four canonical distribution shifts introduced Section Below explain how the statisties captured by descriptor disentangle them.
27

---

Feature distribution shift (P(X)), This shift arises when the marginal distribution of features varies across clients. In our case, it is detected via statistics computed in reduced latent space sfxr), whose geometry satisfies Requirement (R1) (see Appendix C. for implementa- tion details and justifications). Shifts P(X) alter the global mean and covariance of these
latent representations. Since our descriptor includes (1r (k) >xx)), ) clients with similar P(X) naturally map to nearby points in the descriptor space (see Feature-space mean symbol in
Figure 1).
Label distribution shift (P(Y)). When label proportions differ across clients, the overall feature mean is skewed toward the predominant classes. These shifts are likewise reflected in (us >=(h)) allowing label-imbalanced clients to cluster separately. For instance, in a client with few instances of label and many instances of label 2, the mean of (xx(()) will be predominantly influenced by label 2, resulting in proximity to the average representation of label 2 in the latent space. Figure shows how these means in feature space reveal distinct
client clusters.
Concept shift (P(Y This shift arises when different clients assign different labels to identical inputs. Such discrepancies are invisible to feature-only statistics, as the latent representations may be indistinguishable. Figure 1 illustrates this with overlapping means between clients. To resolve this ambiguity, our descriptor includes class-conditional mo- ments {(ua) > ) which enable separation of clients with divergent conditional distributions.
Concept shift (P(X)Y)). This shift occurs when inputs vary across clients for the same label-for for example, the same digit rendered in different colors. In ANDA (Appendix B.1), MNIST digits may share the same label but differ in appearance between clients. These vari- ations alter (pz (k) I!(²), ), and-excepl under rare conditions of perfect latent symmetry can detected without label information.
Because P(X, = P(Y) X) X)P(), capturing both the marginal moments of P(X) and the class-conditional moments of P(Y X) is sufficient to characterize all four shift types. At test time only the marginal component is available (labels are absent), so shifts that rely on P(Y X)) cannot be detected online-a limitation inherent any label-agnostic method, yet fully compatible with our clustering protocol.
D Clustering Mechanics
D.1 Dynamic Clustering Initiation
In this study, we extract data descriptors from the initial trained global model the federation. Due to the inherent high heterogeneity among client data, global model trained using traditional FedAvg typically exhibits limited accuracy. To address this, the clustering process must be initiated carly during training. However, the quality of descriptors improves with a more accurate global model, as additional training rounds refine its representation of the underlying data. To balance these competing factors and ensure the utility of FLUX in real-world applications, we dynamically determine the optimal time to initiate clustering during training.
To analyze the trade-offs involved, we analyze the impact of the training round at which clustering is initiated. Specifically, we tested FLUX by varying the clustering round from to on the MNIST, FMNIST, and CIFAR-10 10 datasets, with the non-lID levels fixed as follows: Feature distribution shift includes rotations (0º, 180%) and three distinct colours; Label distribution shift restricts the number of classes to 3 with bank size of 5; and P P(X concept shift applies augmentations limited to rotations (0%, 90% 180", 270") across 2 classes. As shown in Figure 8, initiating clustering too early results in inadequate representations. while delaying it excessively reduces the time available for cluster-specifie models to converge. Based on this analysis, we establish that clustering should occur no earlier than three training rounds and no later than 80% of the total training rounds to allow sufficient convergence time for cluster-specific models.
To further optimize this process, we leverage the observation that the accuracy of the global model typically increases rapidly in early training rounds before plateauing with diminishing improvements. This behavior is evident in the derivative of performance metrics, such as accuracy, as shown in
28

---

Figure 9. Consequenty. we dynamically initiate clustering when improvements in the global model's accuracy become negligible, formalized as:
c(r) - {
dA ir A (min dF (3gmep) < T V 2 0.8R)
(14)
0
otherwise
Here, represents accuracy, R the total number of training rounds, and T is threshold determining negligible improvement. Through experimentation, we identified T = 0.06 an effective threshold, as varying T did not lead to significant differences in performance, as shown in Figure 10.
This dynamic approach ensures that clustering is initiated at point where the global model is sufficiently accurate extract meaningful descriptors while leaving ample time for the convergence of cluster-specific models.
Model Prefictaace on MNIST
Model Pefforaace on FMNIST
Model Peffomiaze on CIFARI 201
6:06
2.77
0.51
104
1.55
10.00
100
L335-
0,313
0.312
600
050
0.30
Closkiing Rousd
Chankeiin Round
Cloxcing Bous!
Figure 8: Impact of clustering initialization round on model performance. Mean accuracy and standard error are shown across P(X), P(Y), and P(X[Y) shifts for MNIST, FMNIST, and CIFAR-10 10 datasets. Initiating clustering too early results poor representations, while initiating too late limits convergence time for cluster-specific models.
Menn, Ausunees Bourdss MNBST
Meen Accuncy Beands 23NNSS
Bourdh- CIPARIO
Sian Aucaney
Maan Anintay
610
18
Tidedag
Mara Detistove Aessaac XISIST
Mais Deiittive of Aoneary FXNNST
Moan: Derivativn of Artonayy CIFATI
1.3
Mos Deroatze
Denutto
6.70-
Chasarr Aucrmaay
Change
Charga t0
0.00
Todkkng
Tricing Rocnis
Figure 9: Accuracy trends and their derivative during FedAvg training. The figure shows the mean values and 95% confidence intervals across P(X), P(X and P(Y) shifts for MNIST, FMNIST, and CIFAR-10 datasets. The derivative highlights the diminishing accuracy improvements after the initial training rounds.
D.2 Unsupervised Clustering Setup
For the unsupervised clustering 2/ in FLUX, we design an adaptive, density-based -baed method that extends DBSCAN by automatically estimating from the data's 's density distribution and robustly handling outlier clients. To compute the radius parameter e DBSCAN, we first calculate the distances to the second-nearest neighbors for all data points. These distances are sorted in ascending order, forming a curve that reflects the density distribution of the data. To determine the optimal 6, we identify the "elbow point" on this curve which marks the transition from dense to sparse regions in the data,
29

---

Midll Nriemmor MNIST
Pelemraoo CIPA3-10
1.5.0
1843
1943
1945
8.20
4.23
13075
1:00
110
6:274
01
03
Tarobadd
Figure 10: Sensitivity analysis of threshold 7 for clustering initiation. The plot shows mean accuracy and standard eπror across P(X), P(Y), and P(X YY) shifts for MNIST and CIFAR-10 datasets. Results demonstrate that varying I' does not significantly affect performance.
providing a natural boundary for clustering!. To further refine the value and adjust the sensitivity of the clustering. we scale the elbow point distance by a scaling factor. This scaling ensures that the resulting is appropriately calibrated to the characteristics of the dataset. In our experiments, we set the parameter e to 1.0 (indicating no scaling) for the MNIST, CIFAR-1O, CheXpert, and Office-Home datasets. For the FMNIST and CIFAR- was slightly reduced to 0.95 and 0.98, as we observed lower variability in the latent representation. Using the dynamically computed c, DBSCAN is executed with minimum sample threshold of 2, allowing the identification of clusters with as few as two points, Noise points, which are not part of any cluster, are reassigned to individual clusters to ensure that all clients are represented in the analysis. This approach preserves the integrity of the clustering while accommodating the unique characteristics of all clients' data.
For experiments with FLUX-prin -pioor, we use the K-means clustering algorithm, leveraging prior knowl- edge of the number of distributions M, which is used as the number of clusters K.
D.3 Empirical Illustration of Clustering
To illustrate the clustering process in FLUX, provide examples on datasets (MNIST and CIFAR-100 -00) under different levels of heterogeneity. Figure 11 visualizes client descriptors projected into a 2D space via PCA (for illustration only) across three heterogeneity levels (M = 3, 4, 5). In all cases, clients with similar data distributions cluster together in descriptor space, reflecting the latent distributional structure of the datasets and facilitating the grouping process. The figure also shows, with different colors, the cluster assignments produced by our unsupervised algorithm.
It is important note that clustering is performed in the full descriptor space, not in two dimensions. This explains why, for instance, in CIFAR-100 with M = 5, clusters 1, 2, and 3 are not visually separable in the PCA projection, although clients correctly assigned in the higher-dimensional space. Overall, these examples confirm that the descriptors capture essential distributional properties, enabling FLUX to form accurate clusters even under substantial heterogeneity.
Extension to sparse soft assignments. To demonstrate the flexibility of FLUX descriptors beyond hard clustering, evaluate sparse soft variant (Ff. In this setting, hard clustering is replaced with soft-weighted aggregation scheme: each client receives a distribution-specine update by weighting contributions from other clients according to descriptor similarity with its own distribution. Table 6 re- ports averaged results on MNIST, covering all four types of distribution shifts and three levels of heterogeneity for each. The results show that FLUX-Soft substantially outper- forms all baselines improving over the perforrnngg baseline (CFL) by nearly 11 percentage points while proaching the accuracy of hard FL.UX. As expected, our experimental setting hard clustering (FL.Ux) excels, since the data exhibit clear distributional boundaries; however,
Method
Accuracy ( (%)
FodAvg
71.9 6.0
IFCA
35.2 + 6.2
FedRC
75,2 1 4.3
FedEM
75.0 11 4.2
FedSEM
66.0
FedDrift
54.3 7.7
CFL
75.5 4.0
PFedMe
55.2 6.9
APPL.
72.8 5.2
ATP
74.6 8.0
while ap-
FLUX-soft
86.5 1.3
FLUX
89.5 - 4.9
Table 6: Average accuracy on MNIST across shifts and heterogeneity levels.
30

---

MNIST - Optimal Clustering (M-3)
MNIST MANIST-Opiinn Optimal Clustering (M-4)
MNIST Optimal Clustering (M-3)
Cluster
Custtr
-12
-0.1
-18-
-0.2
-12
-0.
-0.
0.0
0.4
-1.2
-0.6
0.0
0.6
1:2
15
-1.0
-05
0.
PC1
PC1
PC1
CIFAR100 Optimal Clustering (M-3)
CIFARI00 Optimal Clustering (M-4)
CIFARIO0 Optimal Clustering (M-S)
5.
4.5
Cluster
Cluster
30-
0.0
15-
-24
Cluster 0.0
-15-
-74
PC1
Figure 11: Visualization of client descriptors projected into 2D space for MNIST and CIFAR-10 100 under different numbers underlying clusters (M = 3, 4, 5). Clients with similar data distributions are mapped close to each other, confirming the effectiveness of FLUX in capturing latent heterogeneity.
FLUX-soft may be advantageous in scenarios with overlapping or noisy distributions that require a more client-centric approach.
Extension to dynamic scenarios. Thanks to modular design, FL.UX can seamlessly extended to non-stationary settings by periodically updating descriptors and reapplying the clustering process. In this dynamic variant, re-clustering is triggered whenever a distributional drift detected or when clients join or leave the federation. To avoid retraining models from scratch after each re-clustering, each newly formed cluster is initialized with the model from the previously existing cluster whose centroid is closest in descriptor space. This association is obtained by matching the current cluster centroids to the most similar ones from the prior clustering phase. Such a warm-starting strategy enables efficient adaptation to changing distributions while preserving previously learned knowledge.
We validate this approach on MNIST under a dynamic setting with two distribution drifts during training and one during testing (Table 6). while all other settings match those in Table 1. As expected, FL.UX achieves substantially larger improvement over all baselines, since none of the existing methods are explicitly designed to handle non-stationary scenarios (except for Ft.UX-soft). These results demonstrate that FLUX remains highly effective under dynamic conditions, outperforming all baselines by over 14 percentage points compared to the best-performing method, thus showing strong resilience and robustness realistic FL deployments.
E Baselines & Comparative Complexity
E.1 Overview of Baseline Algorithms
In our experiments, we benchmarked nine recent baselines designed to address distribution shifts during training test-time. We implemented six CFL-baed baselines, our method falls in this category, that employ diverse clustering principles in CFL, spanning both hard-CFL approaches [15, 18, 17, 14], soft-CFL approaches [11, 16]. metric-basse clustering [15, 17, 16] and parameter- based clustering [18, 14, 11. Additionally we adoted two recent state-of-thh- methods in PFL, and one TTA-FL method.
IFCA [15]: Assuming the knowledge of the number of clusters, the server initializes and broadcasts M models all clients. Each client evaluates the models locally to identify the one
31

---

minimizing its empirical loss. Clients then train the selected model, and the server aggregates updated models from clients assigned to the same cluster.
FedRC [11]: A soft-CFL method that assumes prior knowledge of the number of clusters M. The server broadcasts M models to each client k; which first optimizes weight vector x(k) E TR determining the contribution of each cluster model, before refining all M models. This iterative process alternates between optimizing the client-specific weights and the cluster models.
FedEM [16]: FedEM assumes the number of distributions M is known. It initializes M cluster models, broadcasts them to all clients, and allows clients to learn a weighting vector x(4) RM to determine the relevance of each model for their data. Using alternating optimization, clients jointly update their weight vectors and the cluster models.
FeSEM [18]: This parameter-based method assumes M clusters are known and initializes M cluster models randomly. After local training. clients are assigned to the cluster with the closest model parameters. The server aggregates updated models for each cluster and distributes only the assigned cluster model back the corresponding clients.
FedDrift [17]: A dynamic clustering framework that begins with one model per client. Based on loss similarities, client models are iteratively merged into clusters. The server continuously evaluates all models on all clients, assigning each to the closest model. New cluster models are created when a significant increase in client loss is detected, ensuring adaptive clustering throughout training.
CFL [14]: This gradient- based hard-CFL. method clusters clients based on the cosine similarity of their gradients. It operates under the assumption that at convergence, client gradients should approach zero. Clients with non-zero gradients beyond threshold are split into separate clusters to reduce gradient differences. The process repeats iteratively until no further clusters are formed.
PFedMe [12]: A personalized FL method based on Moreau envelopes that formulates training as a bilevel optimization problem. Each client solves the local subproblem mingom L(k) (1)(k)
[l - 0| via C. gradient steps, then updates its copy of the global model through proximal step before sending it to the server. This decoupling enables adaptive personalization with controllable computation and strong convergence guarantees.
APFL [13]: APFL augments each client with a global copy and a local model ((**, combined into a personalized model T(k) = ag g(*) + (1 ax) 6. During each round, clients update both 0 and ((() via local SGD, adaptively tuning ag balance shared and local knowledge. Only 0 is sent to the server for aggregation, while g(¹) and (kk) remain local. This design enables effective personalization under non-lID data with minimal additional overhead.
ATP [20]: ATP learns a vector of module-wise adaptation rates a = {a where A is the number of modules, by alternating unsupervised entropy minimization and supervised refinement on labeled source clients, then aggregates a via FedAvg. At test time, each unlabeled client downloads the global model and the full vector a, performs unsupervised entropy- -aasep adaptation on its batch (or online stream) using a, and then predicts with the adapted model. Only and the -imeensnsio a are communicated, enabling efficient personalization under distribution shifts.
E.2
Comparative Analysis of Heterogeneity and Computational Costs
This section provides a comparative overview of the types of data heterogeneity addressed by the evaluated baselines, followed by theoretical analysis of the communication and computational costs of FLUX and all implemented basclines.
Supported types of heterogeneity. Table outlines the types of heterogeneity each method claims to support-4n or has been empirically evaluated on real dataset-including feature distribution shift P(X), the label distribution shift P(Y), or the concept distribution shift P(X and PXY]X). The results indicate that most of the baselines were focused on solving only single form of heterogeneity (e.g., IFCA, FeSEM, FedEM, FedDrift, pFedMe), limiting their applicability real-world scenarios where the exact nature of heterogeneity is commonly unknown a priori. Notably, FedRC adopted feature distribution shifts P(X), label distribution shifts P(Y), and concept shift P(Y]X), which closely aligns with the assumptions in this work.
32

---

SUPPORTED TYPES HETEROGENEITY
COMMUNICATION
COMPUTATIONAL (SERVER)
COMPUTATIONAL COST (CIJENT)
ALGORITHM
COST
NIA
p-aaya
O(Kp)
0(2 paran)
IFCA
P(X)
MyNuuye
O(Kp)
O(1 piroio +M
FISEM
NATURAL
pNagrt
O(MKp)
((1(k) perain)
FEDEM
P(Y)
M Myyo
((MKg)
O(Ms ptrain + (100pp
FUDRC
P(X), POY), P(Y]X)
M MygN
((MKg)
O(M, perain + M (k) pint)
P(YX))
Nuyto
O(Kp)
(0) pirain MM plutt
CFI.
P(Y)))
p NNyae
O(K²p)
0(1 perain)
PÉEDME
POY)
PNuyto
O(Kp)
O(tC 1)s(0)ttaing
P(X), P(Y)
pNayte
O(Kp)
O(2.(k) getrain)
ATP
POY)
A)Woyte
O(K(p+A))
012000) piraing O(elM) ptroinntt +Olte))
(pty)a
0(K))+-000
Table 7: Extended comparison table for heterogeneity and cost among FLUX and FL baselines. P(X): Feature distribution shift. P(Y): Label distribution shift. P(X Concept shift (same features, different label). P(Y X): Concept shift (same label, different features). Noyte: Number of bytes per parameters. M: Number of clusters of the current round. K: Number of clients. L: Length of the descriptors. C: Number of gradient steps for local subproblem in pFedMe. A: Number modules in ATP. p: Number of model parameters. v: Latent representation dimension (fe R R"). s(M); Number of sample for client k. Ftrain; Computational cost forward pass and back-propagation. pini; Computational cost forward pass. U: Clustering algorithm (Equation 8). S--s:: Dimensionality reduction (Equation 6).
Computational costs. Table presents a theoretical analysis of the communication and computa- tional overhead introduced by FLUX, in comparison to standard FL methods and the implemented baselines. We focus on the primary sources of system overhead in a single FL round, i.c., communi- cation cost per client (in bytes) and the computational costs on both the server and the clients.
Communication cost: FLUX introduces minimal additional overhead relative FedAvg. The only modification during communication is the transmission of a client descriptor of length L, which is negligible compared the model update size p. As shown in Section C.1.4, our implementation satisfies Requirement (R3) with a descriptor-to-modee ratio of L/p S 3.5 x 10-3
Computational cost (server): the unsupervised clustering procedure (Appendix D.2) operates on client descriptors and introduces a computational complexity of O log L). which remains negligible compared the O(K)) cost of aggregating model updates from K clients.
Computational cost (client): the descriptor extraction includes a dimensionality reduction step Eu-s (Equation 6) with complexity O(52v) (see Section C.4 for full justification), where is the latent dimension and is the number local samples. For typical values of e E (128, 2048), this results in computational cost of at most 8.2 x 10" FLOPs nnggiiggll when compared to the cost of local training. For reference, even single forward pass (no backward or optimization) using smallest model MNIST exceeds 6.5 x 10 FLOPs, while a full epoch with largest model on CIFAR-100 exceeds 6 x 10 FLOPs.
In contrast, several baselines introduce substantial overhead. CFL methods such as IFCA, FedEM, FedRC, and FedDrift require each client to evaluate and/or train all M cluster models locally, significantly increasing both communication (x.) and computation costs. PFL and TTA-FL methods such as pFedMe, APFL, and ATP involve additional optimization steps-at minimum doubling the client-side compute. These overheads may render these methods impractical in cross- device FL scenarios. While methods like CFL and FeSEM avoid extra communication or client-side computation, they shift the burden to the server. These methods demand increased server-side computations, which can become a bottleneck in scenarios with large models or a substantial number of clients, potentially hindering scalability and performance.
Overall, this comparison highlights the efficiency of FL.XX and practical suitability for large-scale, cross-device FL deployments.
33

---

E.3 Comparison with Descriptor-based Methods
We further compare FLUX with HACCS [66], the closest related CFL method that also employs descriptor-like representations. For fairness, HACCS is evaluated without differential privacy, even though its design requires it. Table 8 reports results on MNIST, FMNIST, and CIFAR-10 under both the known association and test phase conditions for FedAvg, FedDrift, HACCS, and FLUX, while the other baselines are reported in Tables 12-39.
The results highlight three key insights. First, HACCS matches FLUX only under the known associ- ation condition for label distribution shifts (P(Y)), where simple label histograms suffice and no semantic modelling is required. In these cases, competitive results. However, HACCS's reliance on label and pixel-value histograms prevents from capturing semantic and structural properties of the data: pixel statistics cannot discriminate subtle but task-relevant variations in the input distributions. As a result, HACCS underperforms FL.UX by a wide margin P(X), X)Y]X), and P(X shifts. Second, HACCS requires label information inference time to build histograms and assign clusters. This prevents test-time adaptation to unseen clients and explains the sharp performance drop in the test phase, where HACCS falls below FedAvg on CIFAR-10 and FMNIST, and far behind both FedDrift and FLUX. By contrast, FLUX produces label-agnostic descriptors at inference, enabling robust cluster assignments even in the absence of labels. Third, unlike HACCS, which generates large and interpretable histograms whose size grows with the number of labels and bins, FLUX uses compact, fixed-size, non-interpretable descriptors. This design ensures scalability and communication efficiency, while reducing privacy leakage risk since descriptors cannot directly reveal label distributions. Overall, these results confirm that while HACCS is the closest related work exploring descriptor-bann -hapter--aaedddeeeee clustering, its practical performance is poor poor-evee even compared existing baselines such as FedDrift. contrast, FL.UX consistently provides privacy-preserving. preservvne,, and generalizable solution for clustered FL.
loss-based methods such as FedDrift achieve
scalable, robust, privacy-pres
Dataset
Method
P(X)
P(Y)
P(YIX)
P(XIY)
FedAvg
77.8 +
91.9 1.4
66.8 1.4
87.0 2.0
FedDrift
94.1
0.44
92.7 8..1
1.7
MNIST (Known A.)
HACCS
5.2
9.2
7.50
88.4 1.4
FLUX
0.5
96.8 0.4
85.1 = 4.0
92.4 2.4
FedAvg
91.9 2.1
N/A
87.0 2.0
FedDrift
47.7
71.8 3.0
N/A
75.9 + 3.1
MNIST (Test Phase)
HACCS
5.5
6.1
N/A
4.8
FLUX
95.0
96.1 1.2
N/A
90.8 + 2.4
FodAvg
72.8 61.6
2.4
55.4 2.3
72.0 1.8
FedDrift
79.9 0.7
86.0 + 1.8
74.5 + 2.7
82.5 + 1.6
FMNIST (Known A.)
HACCS
66.0 3.0 8.0 6.1
77.8 + 3.2
FLUX
77.6 1.8
85.9 + 1.9
72.2 + 3.1
81.9 2.3
FodAvg
72.8 6..6
N/A
72.0 + 1.8
FedDrift
3.5
6.1
N/A
61.9 + 2.3
FMNIST (Test Phase)
HACCS
4.2
6.55
N/A
60.8 + 2.8
FLUX
7.2
85.7
81.0 + 2.8
FedAvg
22,1 1.2
2.9
28.0 1.0
36.0 + 1.1
FedDrift
1.92
29.0 49.0
37.0 1.1
CIFAR-I0 10 (Known A.)
HACCS
4..4 2.62
24.8 1.3
33.4 + 2.0
FLUX
50.4 3.9
31.7 + 1.3
39.1 + 1.8
FedAvg
22.1 1.2
2.9
N/A
36.0 + 1.1
FodDrift
24.0 1.3
38
N/A
36.8 + 1.3
CIFAR-10 (Test Phase)
HACCS
18.9
3.1
30. 1.1
FLUX
33.3 1.0
46.2
N/A
36.7 2.1
Table 8: Per-shift comparison of FLUX with HACCS, FedAvg, and FedDrift on MNIST, CIFAR- 10, and FMNIST. Results are shown for each type of distribution shift (P(X), P(Y), P(Y)X). P(X(Y)) under both known association and test phase conditions.
34

---

F Additional Experiment Results
F. Scaling Data Heterogeneity Types and Levels
We evaluate the effects of data heterogeneity using four non-llI dataset types across 10 clients, generated with ANDA at eight distinct levels of heterogeneity. The detailed configurations for each type of data distribution shift are summarized in Table 9 and elaborated below:
Feature distribution shift (P(X)): Datasets are augmented with rotations and color transformations. Each client is assigned specific augmentation pattern applied to all local data points (images). The non-III level determines the number of augmentation choices per client. For instance, at Level 5, client selects one rotation from [0", 180") and one color from (Red, Blue, Green] for all images. At Levels to 4, the "Original" color indicates no color transformation, preserving the original image channels.
Label distribution shift (P(Y)) )): Datasets retain data points only for specific classes. For example, at Level 8, each client keeps data points from three classes while discarding the rest. Starting from Level 1, with 10 total classes (e.g., MNIST, CIFAR-10. 10, FMNIST), we define class selection bank of size 5 (e.g.. [[0,2,4), [1,3,9]. [3,4.5], [5,6,71. [6,8,9]] at Level 8) to limit the clustering complexity. Clients select subsets from this bank to decide which classes to keep.
Concept shift (P(YiX)): Datasets swap labels among specific classes. A swapping pool is constructed by selecting a subset of classes, which are permuted assign new labels. For instance, at Level 4, the pool (2,3,5,8) might be permuted to (5,8,3,2). mapping images originally labeled as '2" to the new label '5'. The size of the swapping pool increases with the heterogeneity level, and each client permutes the classes independently and randomly. Concept shift (P(XIY)): Clients apply distinct augmentations data points of the same class. For example, Client A applies O9 rotation to class '5', while Client B applies 180 rotation. Augmentation options are limited to rotations (I0", 90% 180 270") }), and the number of classes subjected to augmentation increases with the heterogeneity level.
mon-IID type P(X)
P(Y)
P(Y)))
P(XPY)
Level
Rocation (69) 180%), Coler (Original)
#Closs
#Swapped Class
aCloss Under Augmentation
Level
Rotation (0 120% 2240)) Color (Onginal)
#Class
eSwapped Class
#Class Under Augmeszation
Leve]
Rocaton (0", 90" 180" 270") Colur (Original)
#Class
#5wapped Class
#Class Under Augmentation =3
1
Recation [0". 72%, 144". 216", 288") Color (Original)
aClass 7
#Swappel Clas
#Claes Under Augmentation
Level
Rocation 10", 180 J, Color [Red, Green)
#Class
#Swapped Class 5
#Cluss
Augmentaiion 5
1evel6
Rocation (0a, 120" 240%). Coler (Red, Blue, Green]
#Clnsa
eSwapped Class 6
aClnss Under Augmentation
Levely
Rocation (6?) 90 180", 270% J, Color [Red, Blne, Green)
#Class
eSwapped Class =
#Class Under Augmestation =7
Levels
Rocation 10". 72", 144", 216", 288"), ). Color (Red, Blue, Green)
#Class
#Swapped Class
#Class Augmentation
Table 9: Summary of non-III data heterogeneity configurations across levels. Each type of heterogeneity corresponds to a specific distribution shift.
Tables 10 to Table 49 present detailed results of FLUX and all baseline methods across various types and levels of distribution shifts. We did not evaluate the performance of test phase on P) concept shift, as this problem is unsolvable without access to labels (refer to the Section 4.2.2 and Appendix C.4 for more details). For the baselines that do not provide a solution for real test phase, we weight all models by the number of clients in the cluster, and use the expectation that weights all model outputs as an estimation of the predicted labels.
We also provide Figure 12, which summarizes of the averaged accuracy across the distribution shifts (i.e.. across tables), showing trends under both the known association condition and the real test phase condition. It shows that FLUX consistently outperforms basclines during real test phase, particularly high heterogeneity, which demand accurate identification and clustering of client data distributions to avoid model degradation. The only cases where baselines surpass FLUX are PFL methods on CIFAR datasets. This is expected in scenarios involving P(Y|X) shifts (e.g., Tables 46-47), where each client may have different label preferences for the same input. In such cases, collaboration can be harmful, and methods fine-tuned on the same client distributions seen at test time (as PFL) naturally gain an advantage. However, these methods fail to generalize to unseen clients (see Table 1 and Figure 12, Test Phase), limiting their applicability in realistic deployments. On CIFAR-10, the relatively shallow backbone can also constrain descriptor quality, highlighting that limited model
35

---

expressiveness may degrade clustering and final accuracy; indeed, the gap with PFL methods narrows when using deeper models (e.g.. ResNet9 instead of LeNet5) on RIFAR-100. Notably, FLUX achieves its maximum improvement on FMNIST, surpassing the best baselines by over 23 pp. Furthermore, FL.UX achieves performance comparable to FI.UX-prior where the prior knowledge of clusters (M) is known. This result highlights the challenge of identifying clusters in real-world settings and validates the coherence our unsupervised clustering approach. is worth noting that most baselines rely on this cluster information (see Table 2), significantly simplifying their clustering tasks. Even under the known association setting, FLUX remains the most robust method across all heterogeneity levels and datasets, demonstrating its utility and generalizability.
the number
MINIST
FMENSS
CIFAR-10
CIPAR-10D
Acaunay (Knoonn Avordattee)
Acourayy Assodaation)
Acounay (Knoon Assostttien)
0/7
Dul
0:3
60
0.3
05
041
22
1.3
8:1
0.3
Acourary (TTet (live))
ACCuraCy ('eet (loae))
2.3
0.1
0.
0.1
Landl
Reterngrancty
Beterggradcyy Land
Ianel
Figure 12: Mean accuracy and standard deviation across heterogeneity levels for MNIST, FMNIST, CIFAR-10, and CIFAR-100, averaged over P(X), P(Y), P(Y[X), and P( shifts. First row: known association condition, where test-time cluster associations are available. Second row: test phase condition, where cluster associations inferred.
non-IID Level
Level
Level 2
Level
Level 4
Algurithm
Known
Test Phuse
Known A.
Test Phuse
Known A.
Test Phase
Known A.
Test Phase
FedAvg
93.07 = 0.50
92.15 = 0.58
89.38 : 1.42
89,80 0.88
84.49 = 2.43
86.31 + 2.77
81.56 2.22
85,32 = 2.17
IFCA
91.45 5.81
89.93 + 3:07
92.52
2.22
85.65 1.55
89.74 =:: 3.27
82.94 = 1.95
83.63 2 6.09
81.22 a 4.36
FedRC
89.27 4.03
85.83 = 2.77
73.82 a 4.29
68.84 a 4.78
53.28 a 10.31
50.14 8.28
40.64 = 4.99
47.95 a 4.08
FedEM
89.24 3.98
85.80 2.78
2.78
73.81 + 4.28
68.83 4.74
53.27 10.32
50.13 a 8.28
40.61 = 5.01
47.92 4.09
FeSEM
93.58 = 1.31
91.69 1.76
91.19 1.83
88.99 + 1.60
88.67 = 2.00
85.63 + 3.09
87.62 = 2.12
84.46 + 2.37
FedDrift
95.75 = 0.44
85.50 = 1.88
94.77 + 1.27
76.28 + 3.01
91.99 2.97
71.72 + 3.46
91.48 = 3.79
66.23 + 3.41
CFI.
93.22 0.43
92.74 + 1.13
89.68 1.40
90.10 0.77
84.95 + 2.13
86.91 + 2,42
82.09 1.96
85.95 1.92
pFedMe
94.84 0.22
NIA
94.97 + 0.28
N/A
95.24 = 0.31
N/A
95.41 = 0.30
N/A
APFL
95.72 0.15
91 1..00 0.96
95.56 0.24
88.91 1,48
95,42 0.29
85,20 = 2,16
95,43 = 0.37
84.01 3.09
ATP
N/A
92.07 0.69
N/A
89.11 0.87
N/A
85.91 #: 2.34
N/A
84,92 1.99
FLUX
95.72 0.38
95.61 = 0.57
95.34 =: 0.62
94:45 1.42
94.88 = 1.16
94.49 0.88
93.54 = 2.68
94.81 1.55
FLUX-ptior
95.89 = 0.29
95.69 + 0.33
95.82 = 0.09
94.83 1.02
95.55 0.56
95.39 = 0.80
95.68 = 0.22
95.93 0.32
Table 10: Performance comparison across non-lII Levels 1-4, summarizing all four types of heterogeneity (P(X), P(Y), P(Y]X), P(X(Y) the MNIST dataset. Known A.: Known Association.
F.2 Scaling the Number of Clients
We evaluate the scalability and efficiency of the proposed approach across varying numbers of clients on MNIST dataset, with the non-III levels fixed as follows: Feature distribution shift involves rotations (10', 90% 180%, 270")). Label distribution shift sets number of classes to 4, with bank size of 5. PX) concept shift uses swapping pool size of 5, while P(X concept shift
36

---

non-11D Level
Level 5
Level 6
Level 7
Level 8
Algorithm
Known
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
TedAvg
80.94 1.53
86.62 = 1.75
76.16 2.38
83.92 + 2.58
72.01 = 1.96
81.12 2.09
69.36 = 2.71
79.36 + 2,90
IPCA
83.64 9.88
79.23 = 8.15
79.58 8.22
74.04 7.79
75.95 + 10.07
68.32 6.13
76.00 9.68
64,22 5.89
FodRC
35.61 4.08
42.96 4.58
30.59
3.50
30.25 = 2.61
36.15
26.15 2:41
30.32 2.84
FedEM
35.85 4.15
42.75 4.55
31.16
36.75
31.18 2.39
35.60 3 2.53
31.10 1.49
3.215
FeSEM
87.37 3.28
82.37 4.89
a.772
80.32 3,96
85,60 3.50
75.99 a 4.77
= 5.17
TedDrift
88,98 2.34
64.39 5.30
87.59
4.00
57.70 5.00
88.53 3.62
$2.91 2.69
89.56 2.42
46.28 3.66
CHL
81.39 1.73
87.14 1.93
76.94
1.87
84.94 1.95
72.45 2.06
81.66 2.14
69.51
2.37
79.52 2.54
PPedMe
95.56 0.28
NIA
95.71
0.29
N/A
95.89 0.16
N/A
96.09 0.19
N/A
APFI.
95.45 + 0.26
85.19 2.24
95.41 0.30 N/A
84.08 1.95
95.30 0.26
80.14 + 2.19
95.37 0.39
78.96 2.72
ATP
N/A
86.80 1.41
84,99 2.36
N/A
1.92
N/A
79.92 2.58
FLUX
93.44 2.28
93.88 = 3.43
90.53 + 3.14
93.46 2.82
87.77 = 5.23
92,70 2.96
88.40
4.22
92.32 2.25
FLUX
FLUX-prior
95.67 = 0.24
95.58 + 1.12
95.73 = 0.28
95.73 = 1.88
95.91 0.18
95.78 = 1.36
95.93 0.32
96.28 1.33
Table 11: Performance comparison across non-llD Levels 5-8, summarizing all four types of heterogeneity (P(X), P(Y), X(Y)X), P(X(Y)) on the MNIST dataset. Known A.: Known Association.
non-1ID Level
Level
Level
Level
Level 4
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAug
86.29
= 1.07
4.20
73.18 2.94
73.18 2.94
IPCA
78.79
+
64.70
FedRC
5.99
77.68
4.40
43.07
40.82
FedEM
2:26
43.05
6.80
6.55
FeSEM
88.51
85.01
2.23
80,42 2.44
82.25
82.04 3.24
2.73
FedDrift
0.12
64.87
4.58
95.50
2.61
86.98
0.56
0.81
75.95
2.68
PFedMe
0.09
94.
NIA
APFL.
95.11
82.97 1.23
94.74400.35
79.44 + 2.60
94.76 0.33
73.21 3.74
+ 0.29
70.36 4.81
NIA
2.60
78.77
FLUX
95.96 0.16
95.48
0.76
96.09 0.19
96.03 = 0.26
96.20 0.23
96.20 0.23
95.86 0.24
95.85 83 0.24
FLUX-prior
96.40 0.14
96.40 0.14
96.24 a 0.07
96.24 0,07
96.26 0,23
96.26 0.23
95.88 0.19
95.88 0.19
Table 12: Performance comparison across non-llD Levels 1-4 of P(X) on the MNIST dataset.
Known A.; Known Association.
mon-IID Level
Level
Level 6
Level 7
Level 8
Algorithm
Known
Test Phose
Known
Test Phuse
Known A.
Test Phase
Known
Test Phuse
FedAvg
83.16 0.65
83.16 0.65
78.67 2.14
78.67 + 2.14
73:25 + 2.15
73.25 2.15 2.15
71.35 3.52
71.35 + 3.52
IFCA
86.07 + 18.86
67.56 13:55
82.46 9.90
64.44 10.15
79.63 13.11
58.81 + 8.8
6
58.70 8.04
FedRC
30.51 + 7.26
30.51 + 7.35
14.78 4.88
14.75 4.82
11.40
11.40
11.35 0.00
11.35 0.00
FedEM
30.47 + 7.26
30.77 7.35
14.79 4.89
14.76 4.83
11,40
11.40
11.35 + 0.00
11.35 0.00
FeSEM
86.82 a: 3.88
76.47 7.83
88.19 4 1.38
74.07 a 3.48
2.31
65.76 5.29
89.21 a 2.27
64.79 a 5.49
FedDein
92.86 4: 3.39
51.55 4 6.53
92.51 2.39
46.83 e 5.48
91.50 2.62
48.02 2.13
93.64 1.64
42,80 5.34
CFL
84.06 0.52
84.06 0.52
79.60 1.90
79.60 1.90
73,40 2.23
73,40 + 2.23
72.00 3.44
72.02 3.44
PFedMe
0.21
94.86
N/A
0.09
APFI.
0.27
79.86
94.58
78.67
+ 0.27
2.05
70.68 3.54
87.47
NIA
N/A
78.39
FLLX
95.50 0.10
92.95 5.01
95.35 0.17
95.34 0.18
93.99 + 2.46
93.98 + 2.45
95.26 0.10
93,96 2.60
FLUX-prior
95.35 + 0.26
95.35 + 0.26
95. 12 0.28
95.13 13:1+++ 0.25
95.07 + 0.18
93.38 2.09
94.90 + 0.38
94.92 0.37
Table 13: Performance comparison across non-lIID Levels 5-8 of P(X) on the MNIST dataset.
Known A.: Known Association.
37

---

non-1ID Level
Level
Level
Level
Level
Algorithm
Known
Test Phase
Known
Test Phase
Known A.
Test Phase
Known
Test Phase
FodAvg
95.85 = 0.03
95.85 0.03
94,80 0.36
94.80 0,36
93.98 + 0.62
93.98 0.62
92.32 1.18
92.32 L18
IPCA
96.01 0.14
96.01 0.14
94.99 0.69
94.33 = 0,80
95.23 0.39
93.77 1.77
88.51 7.92
90.18
FedRC
90.21 3.28
90.21 3.29
87.75 1.85
87.76 1.87
86.96 3.13
86.95 3.13
86.78 1,32
86.78
FedEM
90. 20 3.28
90.20 3.28
87.73 1.86
87.73 1.86
86.96 3.15
86.96
3.16
86.65 1.46
86.63
FeSEM
95.
95,94 0.26
94.75 0.93
94.59 0.70
94.54 0.91
93.21 2.18
94.27 1,73
90.93 2.37
FedDrin
0.31
0.42
0.70
77.39
PeedMe
N/A
APFT.
94.64
ATP
N/
95.69 2 0.08
N/A
95.09 = 0.30
NIA
94.62 = 0.36
N/A
94.21 0.82
FLUX
95.52
95.41 0.55
95.50 0.43
93.81 1.29
96.09 0.21
94.94 1.09
96.27 0.63
10 1.07
FLUX-prior
Table 14: Performance comparison across non-IlI Levels 1-4 of P(Y) on the MNIST dataset.
Known A.: Known Association.
non-IID Level
Level 5
Level 6
Level 7
Level
Algorithm
Known
Test Phase
Known
Phase
Known
Phase
Known
Test Phase
FedAvg
91,72 1,15
91.72 1.15
90.74 2.71
90.74
89.22 1.87
89,22 1.87
86.83 3.09
86.83 3.09
IFCA
91.65 3,91
86.66 1.85
85,30 8.96
75.18 8.42
80.98 12,90
66.70 4.98
72.75 15.69
54.54 5.97
FedRC
83.78 1,36
83.68 1.40
83.78 3.56
83.60 3.49
84.11
84,09 3.90
68.42 4.09
67.61 4.80
FedEM
84.79 2.07
83.08
1,16
86.05 3.01
82.54
87.84
82.41 3.95
88.24 1,55
73. 5.47
FeSEM
94.80 1.60
88. 18.14
93.55
5.733
91.65 2.37
82,92 5.98
87.03 3.85
76. 6.61
FodDrift
0.73
69.48 5.39
97.62
4.55
0.28
98.45 0.35
41.41 2.95
CFL
92.00 1,18
92.000 1.18
92,00
92.00 1.09
1.62
90.11 1.62
85.58 1.88
85.58 1.88
PRedMe
96.88 0.21
N/A
97.08 0.22
N/A
97.68
0.28
N/A
98.10 0.56
NIA
APFL.
96.69 0.26
91.95 0.81
97.02 0.18
91.39 1.38
97.49 0.28
89.24
97.94+ 0.37
86.99
94.02
N/A
1.06
93.99 0.47
N/A
92.76 1.02
FLUX
97.02 = 0.40
95.95 + 1.70
97.53 0.38
96.11 3.28
00 # 0.17
98.00 0.17
98.35 0..4
98.35 0.42
PLUX-prior
97.03 = 0.33
95.96 1.64
97.68 0.27
96.19 + 3.15
97.96 + 0.15
97.95 0.15
98.46 # 0..5
98.46 0.35
Table 15: Performance comparison across non-IID Levels 5-8 of P(Y) on the MNIST dataset.
Known A.: Known Association.
mon-lID Level
Level 1
Level 2
Level
Level 4
Algorithm
Known
Test Phase
Known A.
Phase
Known
Test Phase
Known
Test Phase
FedAvg
0.87
N/A
IFCA
95.86 0.14
NIA
2.52
N/A
83.27
3.62
N/A
75.44
N/A
FedRC
95.53 + 3,28
N/A
86.96
2.68
N/A
62.06 + 14.42
N/A
18.46
N/A
FedEM
95.52 + 3,28
N/A
86.94 2.68
N/A
62.03 + 14.40
N/A
18.46
6.84
N/A
F&SFM
95.92 + 0.14
N/A
N/A
84,10
2.35
N/A
80.26
1.46
N/A
FedDrift
95.67 0.09
N/A
95.06 2 1.68
N/A
85.41 5.67
N/A
80.55
7.40
N/A
8.44
N/A
2.09
PFedMe
94.62 0.33
0.37
94.58
N/A
= 0.35
APEL
95.83 1 0.22
N/A
N/A
94.69 0.20
N/A
94.32 à: 0.37
N/A
ATP
95.69 + 0.08
N/A
0.84
N/A
82.09 1,50
N/A
76.27 = 3.01
N/A
FLUX
95.36 0.49
N/A
95.21 0.16
N/A
94.27 1.70
N/A
88.64 4: 4,99
N/A
FLUX-prior
95,93 + 0.05
N/A
95.72 0.08
N/A
94.31 2: 1.06
N/A
94.53 = 0.18
N/A
Table 16: Performance comparison across non-lID Levels 1-4 of PX) on the MNIST dataset.
Known A.: Known Association.
38

---

non-lll) Level
Level
Level 6
Level
Level 8
Algorithu
Known A.
Test Phase
Known A.
Test Phase
Known A.
Phase
Known
Test Phase
FedAvg
52,90 =
N/A
1.53
39.38
IFCA
68.15 3.42
N/A
63,80 4 8.91
NAA
58.20 7.93
N/A
$6.59 4 8.67
N/A
FedRC
13.46 2.27
N/A
10.86 a 0.54
N/A
2:90
N/A
12.82 à: 2.33
N/A
FedEM
13,45 +.25 2.27
N/A
10.86 = 0.54
N/A
12.53
N/A
12.82 2.32
NIA
FeSEM
4.81
N/A
68.62 6,90
NIA
5.778
N/A
66.21 9.16
NIA
FedDrift
1.26
N/A
72.42 6.65
64.16 0,90
N/A
52.92 1.60
N/A
44.81 1.80
N/A
39.49 = 1.75
NIA
PFedMe
94.41 + 0.41
N/A
NA
94.46 0.12
N/A
94.50 + 0.04
N/A
APFI.
94,20 + 0.32
N/A
93.64 = 0.54
N/A
93.28
N/A
93.20 + 0.46
N/A
ATP
70.45 0.53
N/A
61.45 0.47
N/A
54.87 1.57
N/A
48.92 1.72
N/A
FLUX
85.62 4.26
N/A
79.74 4.59
N/A
70.09 8.83
N/A
70.02 + 7.70
NIA
FLUX-prior
94.18 0.16
N/A
94.01 0.21
N/A
94.07 0.14
N/A
93.94 0.20
N/A
Table 17: Performance comparison across non-IID Levels 5-8 of P(YIX) the MNIST dataset.
Known A.: Known Association.
non-11D Level
Level 1
Level
Level
Level 4
Algorithm
Test
Test
FodAvg
94,32 0.89
94.32 + 0.89
92,92 1.02
92.92 + 1.02
90.21 + 2.21
90.21 = 2.21
90.47 + 2.04
90.47 2.04
IPCA
94.49 0.58
95.00 2.00
94.57 1.08
9 6.619
92.25 + 1.71
90.34 2.29
92.91 1.73
89.48 2.45
FodRC
89.57 2.74
62.25 6.60
21.03 10.25
21.03 10.24
16.24
16.25 : 2.25
FedEM
89.59 2.72
2.772
62.26 6.62
62.21 16.4
21.03 10.26
21,02 10.24
16.24 2:25
16.25 2.25
FeSEM
94.71 0.52
94.12 1.99
94.33
91.95 1.13
93.79 a 1.15
89.42 2.96
93.92 1.53
88.20
FedDriat
94.93 0.87
95.51 2.45
93.88 1.00
89.75 2.48
92.82 1.07
83.81 4.15
94.14 1.16
81.27 3.19
CFL
94.65 = 0.64
1.87 95.11
92.75
1.03
92.75 + 1.03
90.47 : 2.03
90.47 2.03
90.86 1: 1.65
90.86 1.65
pFedMe
95.08 0.10
N/A
95.39
0.33 0.33
NIA
95.69 0.25
NIA
95.97 0.45
N/A
APEL
95.80 0.08
93.87 1.43
96.30 0.19
92.73 1.40
96.09 0,18
87.75 2,45
96.68 0.37
89.63 3.02
ATP
N/A
95.68 0.46
N/A
95.07 0.47
NIA
93.68 1.17
N/A
93.46 0.99
FLUX
96.03 = 0.31
95.93 + 0.32
94,58
1.15
93.50 + 2.08
92.94 + 1,55
92,33 = 1.05
93.38 + 1.81
92.46 2.45
FLUX-prior
95.82 0.36
95.71 0.45
95.64 = 0.11
94.30 + 1,37
95.45 = 0.22
94.92 = 0.80
95.69 0.17
95.28 0.42
Table 18: Performance comparison across non-lID Levels 1-4 of P(X on the MNIST dataset.
Known A.: Known Association.
non-IID Level
Level
Level
Level
Level
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phuse
FedAvg
84.99 2.74
84.99 2,74
82.34 2.22
82.34 + 2.82
80.89 + 2.23
80.89 2.23
79.91 1.85
79.91 1.85
IFCA
88.68 2.72
83.47 3.46
86.76 + 3.56
82.51 + 2.83
84.99 + 2.18
79.45 + 3.12
87.64 + 2.28
79.43 1.92
FedRC
14.69 + 2.61
14.70 + 2.61
12.96 + 1.16
12.94 + 1.15
12.97 + 1.88
12.98
1.89
12.00 + 1.04
12.00 1:03
FedEM
14.69 a 2.60
14,70
12.96 1.16
12.94 = 1.15
12,96 a 1.88
12.98 a 1.89
12.00 a: 1.04
12.00 1.03
FeSEM
91.72 1.46
82,50 a 2.35
90.98 + 1.49
81.91 = 1.47
91.08 a 2.30
79.28 2.11
90.36 1.75
77.52 2.53
FedDrin
91.76
72,13 3.53
90.26 + 4.02
68. 68.90 4,89
92.06 + 1.08
61.35 + 2.56
92.06 1.45
54.63 1.72
CFL
85.36 3.09
85.36 3.09
83.23 +: 2.57
83.23 =: 2.57
81.47 2.46
81.47 2.46
80.94 2.01
80.94 2.01
pFedMe
96.19 0.28
N/A
96.52 0.39
N/A
96.61 0.13
N/A
96.90 0.13
N/A
APEL
96.14 0.21
83.78 3.63
96.39 0.41
82.18 a 3.38
96.14 0.18
79.78 2.70
96.20 0.46
79.20 2.05
ATP
N/A
90.22 1.51
N/A
89.18 1.50
N/A
87.43 1.40
N/A
89 1.13
FLUX
93.50 1.59
92.73 2.71
89.52 +: 4.27
88. 93 3.61
88 a: 5.04
86.13 + 4.50
89.95 93:55444 3.46
84.67 2.87
FLUX prior
96.21 0.17
95.41 1.00
96.11 0.34
95.88 # 0.78
96.54 0.23
95.99 1.09
96.43 0.32
95.47 2.25
Table 19; Performance comparison across non-llD Levels 5-8 8 of P(X on the MNIST dataset.
Known A.: Known Association.
39

---

non-IID Level
Level
Level 2
Level
Level
Algorithm
Phase
Known
Test
Known
Test Phase
FedAvg
77.23 a 0.71
76.74 0.80
73.34 L88
74.01 = 1.52
68.52 2.27
69.94 a 2.23
65.53 2.46
68.45 2.74
IFCA
79.29 4.26
76,29 1.71
75.76
70.30 a 2.88
75,91 4.51
65.34 5.23
72.70
64.66 5.60
FedRC
1..54
72.92 1.74
69.29 2.87
2.95
61,03 + 4.07
59.32 4.05
54.87 3.71
FodEM
74.53
72.92
69.49 2.87
67.14
4.111
39.35.4.11
54.00 3.67
$5.58
FeSEM
2.57
76.48 1.33
12.25 2.82
71.51 2:45
69.77
71.34 3.112
6.31
FodDrift
79.77
79.81
55.85 1.97
2.10
2:32
PFedMe
78.86
N/A
79.74
80.79 0.97
N/A
81.45 1.10
N/A
APFL
81.36 0.51
78.13 1.12
81.08 + 0.73
74.57 = 1.92
81.21 # 0.99
70.36 2.53
81.04 1.06
68.08
ATP
N/A
76.91 = 0.79
N/A
73.34 = 1.50
NIA
69.43 = 2.58
N/A
67.22 2.33
FLUX
78.89
78.65
79.68
= 1.47
79.54 1.61
79.68 1.96
80.28
81.43 2.30
FLUX-prior
80.36 1.47
81.39 1.53
81.27
82.60
Table 20: Performance comparison across non-lID Levels 1-4, summarizing all four types of heterogeneity (P(X), P(Y), P(YIX), P(X(Y) on the FMNIST dataset. Known A.: Known Association.
mon-IID Level
Level
Level
Level 7
Level 8
Algorithm
Test Phase
Known
Test Phase
Known
Test Phase
FedAvg
66. 14++ 2.50
70.42 2.12 2.12
65, 3.99
67.53 + 4.37
56.77 + 1,92
62.81 + 1.64
54.08 + 3.72
60.61 3.98
IFCA
72.84 4.25
65.12 4.03
71.52 4.336
61.24 = 6.06
66.27 5,93
52.53 4.75
65.18 5.71
52.58 6.80
FedRC
55,22 5.76
46.37 a 8.32
50,08 8.89
39.86 5.17
44.29 4.77
40.83 6.07
FedEM
52.03 5.90
48.04 8.51
50,00 8.65
43.71 + 5.39
44.67 4.92
#6.11
4 5.59
FoSEM
71.46
65.88
71.31
64.23 5.332
59.38
66.16
5.95
FodDrift
80.56
81.60
3.40
82.64 2.86
39.30
82.55 3.00
35.37
57.23 2.98
3.34
61.21 a
pleddMe
82.38
0.74
84.63
N/A
APFL
81.54 1.53
70.55 +
82.06
68.41 3.32
83.47 + 0.92
63.61 2,50
83.82 1,58 N/A
59.90 3.09
ATP
N/A
70.51 = 2.41
N/A
66.65 = 4.77
NIA
6L.87 2,86
61.49 5.07
FLUX
78.90
4.03
82.26 2,98
79.01 4.38
82.05 4.90
80.00 3.06
84.26 3.08
FLUX-prior
1.83
0.79
86.29
1.47
86.71
Table 21: Performance comparison across non-lID Levels 5-8, summarizing all four types of heterogeneity (P(X), P(Y), P(Y[X), P(X(Y) on the FMNIST dataset. Known A.: Known Association.
non-lID Level
Level
Level
Level
Level 4
Algorithm
Known
Test Phuse
Known
Test Phase
Known
Test Phase
Known A.
Test Phase
FedAvg
72.07 = 0.93
72.07 a 0.93
67.35 1.20
67.35 a 1.20
61.17 a 2.66
61.17 a 2.66
57.95 2.57
$7.95 2.57
IFCA
77.29 =: 8.46
68.53 = 2.72
67.37 a 10.47
53.81 4.65
74.36 7.58
44.84 a 8.54
69.11 = 8.72
48.37 2 6.79
FedRC
69.16 1.38
68.47 = 1.46
65.72 1.38
60.29 + 2.22
53.64 6.35
47.41 5.79
50.00 = 5.26
43.32 6.58
FedEM
69.24 = 1.22
68.56 1.18
65.87 = 1.45
60.33 2.34
$3.98 a 6.37
47.46 = 5.78
49.87 = 5.08
43.14 6.51
F&SEM
77.45 4.88
4.83
69.96
1.83
74.26 = 5.03
63.05 4.13
64.98 3.91
60.19 3.34
68.82 3.90
$2.94 + 7.97
FedDrift
81.21 0.27
50,90 0.19
80.94 =: 0.31
34.10 0.24
80.21 0.29
28.18 2.40
80.47 0.34
27.27 1.58
CFL
72.65 1.08
1.08 72.65
68.18 1.22
68.18 1.22
62.64 2.24
2:24 62.64
58.59 2.76
58.59 2.76
pFedMe
79.43 0.58
N/A
78.96 0.28
N/A
79.51 0.20
N/A
78.95 0.07
N/A
APEL
80.86 = 0.12
73.19 1.07
80.01 0.08
66.92 2.74
80.11 0.48
59.59 a 4.47
79.26 0.50
54.71 5.10
ATP
N/A
75.69 0.17
N/A
70.96 0.59
N/A
67.70 1.32
NIA
66.29 1.75
FLUX
79.58 0.81
79.35 = 0.56
78.72 0.68
78.48 0.66
79.63 1 0.47
79.21 1 0.66
79.28 0.39
79.23 2 0.34
FLUX-ptior
80.92 =: 0.23
80.92 a 0.23
80.02 = 0.15
80.02 0.15
80.91 0.41
80.21 = 0.41
79.77 0.15
79.77 0.15
Table 22: Performance comparison across non-lID Levels 1-4 of P(X) on the FMNIST dataset.
Known A.: Known Association.
40

---

non-1ID Level
Level
Level
Level
Level 8
Algorithm
Known
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
67.63 0.62
67.63 0.62
61.41 3.49
61.41 3.49
53.65 1.51
53.65 1.51
51.73 6.16
6.166
JPCA
78.56 = 1.23
53.01 5.08
74.12 3,20
4
65.53 6:55
28.82
71.93 5.07
36.54
FodRC
46.85 8.46
44.72 7.86
30,74 15.17
29.49 14.50
20.83 6.89
20.35
18.21 8.04
17.95 7.77
FedEM
46.58 8.59
44.51 7.70
30,80 15.17
29.46 14.43
20.94
18.26 2.34
17.98 4788
FeSEM
72.19 3.81
57.96
71.27 + 3.53
$2.52
67.47 2.64
44.47
67.39 3.57
39.63
TedDrift
79.78 0.57
30.69 2.86
79.59 0.33
24.53 1,47
78.07
22.05 1.64
77.49 2.55
22.41 1,59
CHL
68.41 0.98
68.41 0.98
62.63 3,20
62.63 + 3,20
53.67 5.01
53.67 5.00
53.38 5.13
53.38 5.13
PRedMe
79.21 0.40
N/A
79. 0.49
NIA
79.00 0.30
N/A
79.08 0.21
NIA
APEL
79.65 0.14
65,90 1,53
79.47 0.45
60.77 5,49
79.52 9.41
52.10
78.26 a 0.86
$2.53 5.01
ATP
NIA
70.91 1.40
N/A
66.86 2.77
N/A
N/A
64.13 a 3.75
FLUX
78.63 = 0.67
77 2.40
76.78 2.29
76.38 + 2.77
73.92 5.15
71.30 + 6.07
14.37 3.94
74.32 +: 3.89
FLUX-prior
78.75 1.00
78.72 1.04
78.23 0.53
78.19 2 0.51
7.11
78.09
77.66 0.74
77.66 0.74
Table 23: Performance comparison across non-III Levels 5-8 of P(X) on the FMNIST dataset.
Known A.: Known Association.
non-ITD Level
Level
Level 2
Level
Level 4
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
0.23
74.81 a
75.07
ITCA
80.64
80.77
77.20
FodRC
75.03 0.28
15.03 0,26
7.154
73.53 2.11
70.28 2.38
70.25 : 2.37
68.21 1.73
68.19 1.79
FodEM
74.98 a 0.29
74.98 a 0,28
73:46 2.11
73:44 2.11
70.22 i 2.37
70.20 2:40
68.26 1.68
1.68
68.10 1.81
FiSEM
80.33 a 0.43
78:20 2.00
78.69 e 2.11
77.02 a: 2.08
76.12 2.15
76.61 2.93
71.85
2.03
FedDrin
79.14
7 0.17
77.89 1.51
76.24 = 2.94
81.90 1.63
69.68 & 1.71
83.25 a 1.26
62.63 2.14
CFL
79.05 1:055:000 0.13
79.05 + 0.13
78.21 + 1.68
78.21 = 1.68
75.24 + 2.66
75.24 + 2.66
75.29 #: 1.63
75.29 1.63
pFedMe
37.75 : 0.68
N/A
80.85 + 2.03
N/A
83.10 + 0.90
N/A
84.72 # 1.93
N/A
APEL
80.81 0.32
81.01 0.65
81.82 + 1.16
79.08 = 1.55
83.08 + 1.62
77.53 + 2.28
84,50 + 1.52
74.66 1.79
ATP
N/A
+
NIA
79.06 = 1,16
N/A
78.21 2.27
N/A
78.76 1.22
FLUX
77.37 a: 0.46
76.91 0.46
80.69 a 1.41
79.70 = 1.08
81.05 +: 1.94
80.77 a: 2.01
84.09 1.55
83.83 a: 1.74
FLUX-prior
78.01 + 0.39
78.00 + 0.46
80.88 + 1.45
80.06 # 1.59
81.75 + 1.89
81.15 1.91
84, 34.48 + 1,36
84.30 + 1.57
Table 24: Performance comparison across non-IID Levels 1-4 of P(Y) on the FMNIST dataset.
Known A.: Known Association.
mon-IID Level
Level
Level 6
Level 7
Level 8
Algorithm
Known
Test Phase
Known
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FodAvg
73,83
67.33 1.87
1.87
63.11
IFCA
73.58
81.17
7.03
64.04 4.52
4.55
a
FedRC
69.97
65.84
65.08
FodEM
70.50 4.80
68.77
76.97 5.63
69.91 3.00
81.22 3.90
66.26 4.04
80.67 6.80
64.93
FeSEM
4.75
71.02 5.37
80.89
72.31 = 4.48
78.50 8.17
67.89 4.81
72.53 1..58
65.51 2:93
FodDrift
86.93 2.51
60.42 1 4.92
55.94 5.54
93.63 1 0.90
47.12 3.98
95.13 2.50
39.98 3.03
CFI.
18 3.48
74.18 3.48
70.99 + 8.39
70.99 + 8.39
67.47 + 1.50
67.47 + 1.50
62.69 2.38
62.69 2.38
PRedMe
87.44 1.75
N/A
88.42 2.48
N/A
92.84 + 0.95
NIA
93.55 + 2.81
N/A
APFL
85.65 + 3.13
3.32 75.08
87.55 + 4.80
74.69 3.33
91.40 + 1,10
69.98 + 2.14
93.04 4,41
59.33 3.45
ATP
N/A
79.06 3.67
N/A
77.94 5.93
N/A
78.76 3.65
NIA
78.47 6.03
FLUX
86.38 2 2.64
86.48 2.53
+ 2.96
89.65 3.06
93.54 + 1.04
93.55 1.00
94.72 2.82
94.74 2.79
FLUX-peioe
86.95 2.12
86.93 = 2.12
89.97 :: 3.01
89,97 = 3,01
93.71 + 1.03
93.61 = 1.03
94.80 2.73
94.80 2.73
Table 25: Performance comparison across non-IlI Levels 5-8 of P(Y) on the FMNIST dataset.
Known A.: Known Association.
41

---

non-lID Level
Level
Level 2
Level
Level 4
Algorithun
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known
Test Phase
FedAvg
78, 69 0.32
N/A
71.32 = 2.69
N/A
64.28 2.39
N/A
56.78 1.30
NIA
IFCA
78.70 0.27
N/A
74.47 4,00
N/A
67.91 4.42
N/A
64,90 2.57
N/A
FedRC
NIA
N/A
FedEM
0.27
N/A
2.87
N/A
N/A
46.11 = 1.70
NIA
FeSEM
78.71 = 0.27
N/A
74.09 #.88
NIA
66.84
1884=199
N/A
63.26 = 1.14
N/A
FedDrift
79.06 0.14
N/A
76.86 + 2.99
N/A
74.52 3.63
N/A
74.70 + 0.90
N/A
CFI.
79.03 0.14
N/A
71.66 2.78
N/A
64.69 2.16
N/A
57.17 1.05
N/A
PFedMe
77.94
N/A
7
77.84
NIA
APFL
0.32
79.41 = 73.33 = 1.48
0.52
77.05 +
ATP
79.13 0.27
N/A
N/A
67.17 # 1.99
N/A
61.58 + 0.65
N/A
FLUX
77.43 0.37
77.85 #: 0.56
N/A
77.85 0.44
N/A
75.33 4: 2.13
N/A
FLUX-prior
78.86 0.19
N/A
78.80 a 0.30
N/A
76.98 1.28
N/A
77.10 0.32
N/A
Table 26: Performance comparison across non-IlD Levels 1-4 of PX) on the FMNIST dataset.
Known A.: Known Association.
mon-llI) Level
Level
Level 6
Level
Level 8
Algorithm
Phase
Test
FedAvg
2.53
38.65
IFCA
51.57
5.47
N/A
50.23 : 2.26
N/A
45.19 4.88
N/A
44.26 2
NIA
FedRC
38.85
4.49
NIA
33.77 + 4.55
N/A
25.31 = 6.46
N/A
22.40 4.42
N/A
FedEM
4..50
N/A
33.78 4.55
N/A
2
N/A
22.40 4.42
N/A
FeSEM
61.14 2.62
NIA
57.73 4.99
N/A
55.22 3.65
N/A
52.56
N/A
FedDrift
72.43
N/A
73.31 2.02
5.220
N/A
72.80 4.16
N/A
46.62 2.50
39.00
N/A
pFedMe
77.61 0.35
N/A
77.63 0.20
NIA
77.29 + 0.27
N/A
77.77 + 0.23
N/A
APEL
76,26 0.60
N/A
76.13 0.288
N/A
76.39 = 0.35
N/A
77.41 + 0.68
N/A
ATP
58.71 3.03
N/A
$2.65 + 2.79
N/A
48.16
1.91
N/A
43.68 + 2.50
N/A
FLUX
69.37 10.37:55000
N/A
68.26 1 6.41
N/A
64.61 = 6.32
N/A
66.84 3.27
N/A
FLUX-prior
76.96 0:25
N/A
77.03 = 0.46
N/A
76.77
N/A
76.53 + 0.33
N/A
Table 27: Performance comparison across non-lID Levels 5-8 of P(Y]X) on the FMNIST dataset. Known A.: Known Association.
non-IID Level
Level
Level
Level 3
Level
Algorithm
Test
FedAvg
79.39
1.00
79.39
1.00
76.90 1.29
76.90 1.29
73.84 1.34
73.84 1.34
4.774
72.32 3.74
ITCA
80.53
0.84
79.60 1.03
80.43 + 0.92
77.11
1.00
79.67 + 0.85
73.04 2.05
79.59
71.35
FedRC
75.26 2.65
75.27
2.62
67.46 4.03
67.47
4.05
60.33 3.18
60.31
3.17
4.665
55.25 4.69
FedEM
75.22
75.21 2.62
67.69 4.26
67.65 4.27
60.40 3.45
60.40 3,46
55.36 4.772
55.31 4.69
ReSEM
75.00
70.56
FedDrin
81.90
77.12
74.51
pFedMe
80.49
N/A
81.14 1.38
N/A
82.44 2.25
N/A
84.30
N/A
APEL
82,97
1.28
80.18 1.64
82.89 1 0.77
77.72 = 1.47
83.03 + 1.32
73.95 a 0.84
83.32 1.34
74.88 1.86
ATP
N/A
80.37 a 0.87
NIA
79.27 = 0.90
NIA
77.57 1.79
NIA
77.18 1.81
prior
Table 28: Performance comparison across non-IID Levels 1-4 of P(X (Y) on the FMNIST dataset. Known A.: Known Association.
42

---

non-IID Level
Level
Level
Level 7
Level 8
Algorithm
Phase
Known
Test
Known
Test Phase
FedAvg
69.80 0.97
69.80 0.97
69.28 1.99
69.28 = 1.99
67.44 1.50
67.44 1.50
66.99 2.57
66.99 2.57
IFCA
79.48 3,05
68,78 1.97
80.55 2.29
67.10 2.47
78.89 4,93
64.73 2.59
72.93 7.25
63.79 3.25
FedRC
52.00 4.87
51.97 a 4.89
50.97 +: 2.48
50,91
2.49
47.46 2.58
47.44
46.27 +.37
FodEM
$2.20 4.70
4.71
$0.64 2.65
50.63 2,66
47.36 2.86
47.36
45,90
45.88 4.06
FoSEM
75.18
68.66
2.17
1.06
73.67 + 1.84
65.76 1.14
72.15
4..59
FodDrift
= 1.34
1.74
48.73 2.51
84.79 2.44
4
70.47
=
1.39 N/A
2:43
PFedMe
N/A
87.67 * 1.45
88.10 0.36
N/
APFL
84.62 2.26
70.66 #
85.09 1.03
69.78 = 1.15
86.58 1.83
68.75 2.51
86.57
0.35 N/A
67.83
ATP
N/A
16.20 =
N/A
75.99 = 1.24
N/A
75.09 1.36
74.25 0.58
FLUX
81.24
81.26
3.11
83.98 3.02
81.30
84.06 1.82
83.73 2.37
FLUX-prior
0.83
87.34 1.16
1.24
87.67 0.74
87.67
Table 29: Performance comparison across non-IID Levels 5-8 of P(X (Y) on the FMNIST dataset.
Known A.: Known Association.
non-1ID Level
Level
Level
Level 3
Level 4
Algorithm
Known
Test Phose
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
0.19
36.33 0.19
34.90 1.08
34.92
34.19
38. 1.37
33,16+
34.81
IFCA
40.01
39.96 1.29
39.37 1.78
38.99 =
38.56 1,91
+ 1.41
39.22 28.66+2.13
FodRC
24.88 #
20.52
17.82 2.33
16.95 +
17.23 3.66
14.66
15.94 = 2.67
FodEM
24.84 2.01
20.44
21.29
17.82 2.48 2.48
16.97 3.56
17.22 + 3.65
2.50
15.96 = 2.69
FuSEM
37.65 0.83
38.33 0.79
37.23 1.23
37.52 1.16
37.52 + 1.47
37.85 + 1.46
2:43
37.68 3,00
FedDrift
38.08 0.88
37.27 + 0.93
36.28 1.18
35.51 + 1.42
35.40 + 1.07
36.70 2.01
34.98 + 1.74
34.16 2.15
CFL
37.83 + 0.34
0.61 37.39
36.04 + 1.11
36.04 # 0.98
35.27 + 1.11
36.20 + 1.20
34. 10 2.29
35.68 2.62
PFedMe
38.28 + 1.07
N/A
22.6 0.88
N/A
40,41 + 0.83
N/A
41.71 + 0.87
N/A
APEL
# 0.62
41.13 1.23
43.01 = 0.66
38.93 = 1.08
43.45 + 0,71
38. 0.85
44.44 + 0.81
38.55 1.18
ATP
N/A
36.85 0.65
N/A
36,20 2 1.08
NIA
36.17 + 1.19
N/A
35.71 2.06
FLUX
36.65 a 0.85
36.19 1.17
37.49 1.02
37,06 # 1.11
38.29 4 1.05
37.69 + 1.79
37.82 1.69
38.35 1.76
FLUX-prior
37.39 a 0.82
37.12 a 0.95
37.69 = 1.12
37.17 a 1.40
38.19 a 1.07
37.32 a 1.60
39.52 1.29
38.68 1.89
Table 30: Performance comparison across non-lID Levels 144, summarizing all four types of P(Y), heterogeneity P(Y), P(Y]X), P(X(Y) on the CIFAR-10 dataset. Known A.: Known Association.
non-1ID Level
Level 5
Level 6
Level 7
Level 8
Algorithm
Known
Test
Phase
Known A.
FedAvg
30.37 1.16
31.61 + 1.14
28.07 3,16
29.50 a 3.56
25.30 à 2.93
26.47 a 3.31
24.53 2.59
26.09
2.90
IFCA
37.08 2.05
2.338
36.64
35.08 2.84
38.38
32.87
2.69
3
31.96 3.113 3.13
FodRC
1.33
1.8
15.58 1.89
17.29
15.07
16.37 1.14
2.54 2.276
FodEM
1
16.10 1.87
1:221
16.47 1.65
16.69 1.25
2.07
17.25 2.02
FeSEM
36.28
2.33 2.33
3.100
30.62
FedDrin
1.72
33.25
26.75
34.27
CFL
31.51 1.11
32.87 1.00
29.6
31.36 + 3.85
26,91 + 3.08
28.42 3.48
25.95
27.83 2.52
PRedMe
0.663
N/A
43.78 1.36
N/A
45,49 + 1,23
NIA
47.34
N/A
APEL
44.00 + 0.72
2..34
44.80 1.40
34.44 = 2.85
46,47 + 1.54
32.84 + 2.68
48.45 : L.14
32.66 1.61
ATP
N/A
32.78 à: 1.39
N/A
NIA
29.50 +: 2.76
N/A
29.11 2.73
39.13 2.06
38.60
3.60
FLUX-priio
40.25 1.30
2.13
2.01
2.49
44.28
42,03
Table 31: Performance comparison across non-lID Levels 5-8, summarizing all four types of heterogeneity (P(X), P(YIX), P(X[Y)) on the CIFAR-10 dataset. Known A.: Known Association.
43

---

non-IID Level
Level
Level
Level 3
Level
Algorithm
Known
Phase
Known
Phase
Known A.
Test
Known
Test Phase
FedAvg
33.26 0.21
33,26 0.21
28:68 L.3
28.68 = 1.03
29.97 0.45
29.97 0.45
28.72 0.69
28.72 0.69
ITCA
39.49 1.30
37.27 0.27
35.18 2.26
32.23 = 1.24
34,82 2.35
33,08 1.47
34,45 1.62
32.31 1.02
FodRC
28.35
28.33
18.86
18.87
4.36
15.65 4.34
11.69
11.68
FodEM
28.43 0.79
0.76
38.
15.65
11.68 1.16
FeSEM
31.34
= 1.31
1.69
1.86
32.04
1.42
FedDrift
35.20
33.73 +
30.54+ 1.42
28.00 2.08
30.88 0.29
30.88 0.29
19.19 0.82
CFL
34.11
34.11 0.33
29.25 + 1.03
29.25 = 1.03
30.74 0.39
30.74 0.39
29.21 0.74
29.21 0.74
PFedMe
37.75 0.93
NIA
36.88 +: 0.99
N/A
37,96 0.43
N/A
36.71 0.73
N/A
APEL.
41.02 0.87
37.20 a 1,33
39.06 1.02
31.51 = 0.22
40.15 # 1.05
33.25 # 1.60
38.83 # 1.33
31.42 + 0.46
ATP
N/A
34.07 0.49
NIA
30.47 = 0.95
N/A
31,26 1,36
N/A
30.06 0.47
FLUX
36.73
36.32
+ 0.72
0.22
34.25 0.34
FLUX-prior
37.20
37.20
34.52
# 1.33
35.51 + 0.41
35.46 0.49
34.74 + 0.39
34.74
Table 32: Performance comparison across non-III Levels 1-4 of P(X) on the CIFAR-10 dataset.
Known A.: Known Association.
non-IID Level
Algorithm
Known
Test Phase
Known
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
19.53 a 0.60
19.53 a 0.60
11.76 a 2,02
11.76 = 2,02
12.38 a 1.80
12.38 a 1.80
12.47 2,56
12:47 a: 2.56
IFCA
28.67 0.56
20.71 a 3,27
25.62 2.32
19.77 a 1.62
30.04 a 2,36
18.74 a 1.35
27.46 a: 1.85
17:47 3.23
FedRC
11.90 1.40
11.89 1.38
11,32 1.13
11.33 1.14
10.70 0.89
10.70 0.88
10.66 0.70
10.65 = 0.69
FedEM
11.90 1.40
11.89 1.39
11,32 1.14
11.33 1,15
10.70 0.89
10.70 0.88
10,66 0.70
10.65 0.69
FoSEM
2.51
19.84 a 2.24
20.50 1,93
15.51 2.43
23.72 1.73
18.28 1.99
21.77 2.52
15.97 2.03
FedDrift
22.44 1.89
21.16 1.51
16.79 3.44
15.57 a 1.53
18.08 3.26
16.69 1.88
18.23 a 2.28
16.97 : 1.26
CFL
20.21 0.95
20.21 0.95
15.66 1.36
15.66 1.36
16.43 1.32
16.43 1.32
16.08 1,06
16.08 1.06
PeedMe
35.53 0.73
N/A
34.03 0.80
34.84 0.21
N/A
33.96 0.53
N/A
APEL
0:98
21.43
33.45
18.75:2.. 2.38
34,08 1.18
18.28 3,16
33.82 0.52
18.89
N/A
22.95
22.28 0.83
N/A
21.71
FLUX
33.51 1.07
33.51 1.07
32.52 0.56
32,52 = 0.56
30.34 1: 1.81
30.34 1.81
29.11 2.23
29.11 2:23
FLUX-prior
33.64 0.75
33.71 0.64
32.32 0.68
32.32 0.67
32.70 0.98
32.66 1.05
31.60 0.31
31.60 0.31
Table 33: Performance comparison across non-lID Levels 5-8 of P(X) on the CIFAR-10 dataset.
Known A.: Known Association.
non-IID Level
Level
Level
Level 3
Level 4
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
37.74
17.74
38.41
38.35
IFCA
0.99
43.15
+
2.59
1.74
45.08
FedRC
2
21.24 3.68
22.68 a 1.62
22.69 = 1.60
24.83 a 4.32
24.77 a 4.36
24,83 4:21
24.82 4.18
FedEM
21.26 3.73
21.25 a 3.72
22.71 a 1.47
22.70 = 1.46
24.86 4.33
24.74 a 4.37
24.93 4.32
24.87 4.21
TeSEM
38.68
40.60 a 0.78
40.55 + 1.17
40.72 = 1.42
41.91 + 2.07
41.36 1.61
44,80 44.22
42.39 4.76
FedDrin
39.06 0.24
39.06 0,24
39.65 + 1.10
39.65 + 1.10
40.01 + 1.34
38.14 3.14
43.41 +:7.6
35.75 3.19
CFL
09.0720..2
39.07 a 0,29
39.91 # 1.09
39.91 = 1.09
39.70 1,32
39.70 1.32
39.92 4.06
39.92 4.06
pFedMe
37.89 1.17
N/A
41.56 1 1.12
N/A
44.27 a 0.36
N/A
48.74 0.49
N/A
APEL
42.95 0.59
42,79 a 1.55
45.93 + 0.85
43.44 = 1.85
47,39 a 0.40
41.27 a 0.53
51.27 a 0.79
43.09 1.32
ATP
N/A
38.94 0.18
N/A
40.86 = 1.10
N/A
40.86 1.08
N/A
42.35 + 3.77
FLUX
36.24
35.72 + 0.00
40.54 1.66
39.27 = 1.28
42.68 + 1.61
39.84 + 2.56
46.76 1.86
43.58 + 2.76
FLUX-prior
36.66 0.49
36.17 0.62
40.81 + 0.61
39.46 = 1.03
43.53 + 0.66
39.90
47.59 2.20
44.41 1:.11:00 2.69
Table 34: Performance comparison across non-lID Levels 1-4 of P(Y) on the CIFAR- 10 dataset.
Known A.: Known Association.
44

---

non-IID Level
L
Level
Level
Level
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
41.17
33.02
33.02
52.71
4.29
FedRC
27.15 1.23
27.06+ 1.18
29.38 3924
27,10
0.99
4:05
FedEM
= 2.97
28.28 2.10
31.46
30,05 + 3.47
32.70 2.47
1.31
39.32
3.04
FeSEM
47.61 2.92
47.61 2.92
10.81
4.111
44.86 5.15
38.43 4.95
41.72 10.78
39.51
FedDrift
41.19
+ 6,49
5,80
34.22 5.80
PRedMe
0.06
N/A
63,35 2.52
72,70
APFL
54,94 = 0.37
44,97 2.37
59.43 2.37
45.32
68.077
74.35
44.27 = 1.71
ATP
N/A
44.65 1.35
N/A
5:432
N/A
39.28 4.88
NIA
44.13 = 6.05
BLUX
$1.59 = 2.23
4
54,88 6.30
5,90
60.67
4.75
6..12
69.75 4.18
56.54 10.14
PLUX-prior
51,65 2.31
47.89 2.66
56,97 3.882
51.39
60.78 4.77
55.22 6.65
69.84 4.14
$7.90 10.37
Table 35: Performance comparison across non-lID Levels 5-8 of P(Y) on the CIFAR-10 dataset.
Known A.; Known Association.
non-IID Level
Level
Level 2
Level
Level 4
Algorithm
Known
Test Phase
Known
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
37.74 0.18
NIA
34,86 # 1,36
NIA
31.39
0.59
N/A
28.19 0,70
N/A
IFCA
37.74 1.17
NIA
37.72 2.23
N/A
34.29
0.94
N/A
33,02 0.96
N/A
FedRC
16.08
3,27
N/A
FedEM
37.70
31.70
16.11
N/A
10.77
FeSEM
37.77 0.22
N/A
37.81 # 1.03
N/A
34.67 = 0.72
N/A
33.81 + 1.10
N/A
FedDrift
39.03 +
N/A
36.04 + 1.31
N/A
32.60
N/A
29.40 0.63
N/A
CFL
0.100
N/A
36.04 1.42
N/A
32.48 0.75
N/A
29.38 = 0.65
N/A
PFedMe
+
NIA
38.05 1.04
NIA
37.78
1.05
N/A
37.71 1.01
N/A
APEL
42.95 0.59
42.23
40.87
0.63
N/A
40.51
ATP
38.94 0.18
N/A
36.47 # 1.44
N/A
33.03 = 0.89
N/A
29.69 :: 0.97
N/A
FLUX
36.11 0.14
=
1.64
FLUX-prior
37.56 0.35
N/A
36.75 a 0.59
N/A
34.78 a 0.73
N/A
34.60 a: 0.70
N/A
Table 36: Performance comparison across non-IID Levels 1-4 P(Y on the CIFAR-10 dataset.
Known A.: Known Association.
non-lII Level
Level 5
Level
Level 7
Level 8
Algorithm
Known
Test Phase
Known A.
Test Phase
Known
Test Phase
Known A.
Test Phase
FodAvg
= 1.24
23.77 1.45
NA
21.81 +
19.87 + 1.28
IFCA
30.02 + 2.64
N/A
29.23 a 2.34
N/A
27.12 + 2.26
N/A
26.50 4 0.86
N/A
FedRC
11.43 1.57
N/A
10.42
N/A
11.17
1.38
N/A
11.15
0.96
N/A
FedEM
11.43 1.57
N/A
1.87
N/A
11.17 1.38
N/A
11.16 0.95
N/A
FeSEM
33.05 + 1.02
NIA
32.62 1.04
N/A
29.40
2.26
N/A
28.75
1.34
N/A
FedDrin
CFL
2.39
N/A
24.50
NA
22.36 + 1.23
N/A
20.29 1 1.39
N/A
PFedMe
+.94
N/A
38.36
1.24
N/A
38.33 + 1.19
N/A
37.67
1.40
N/A
APEL
39.76 1.43
N/A
39.09 + 1.45
N/A
39.08 + 1.55
N/A
38.81 +: 1.64
N/A
ATP
27.53 1.06
N/A
25.06
1.68
N/A
22.97
1.09
N/A
21.26
1.45
N/A
FLUX
32.55 0.67
N/A
26.55 1.05
N/A
28.72 2.27
N/A
26. 86 + 3.86
N/A
FLuX-prrar
34.07 0.53
N/A
33.78
0.73
N/A
33.71 0.78
N/A
33.35 0.58
N/A
Table 37: Performance comparison across non-IIID Levels 5-8 of P(X) on the CIFAR-10 dataset.
Known A.: Known Association.
45

---

non-IID Level
Level
Level
Level
Level
Algorithm
Known A.
Test Phuse
Known A.
Test Phase
Known A.
Test Phuse
Known A.
Test Phase
FodAvg
38.00 + 0.16
38.00 0.16
37.67 0.73
37.67 0.73
37.06 4 1.77
37.06 1.77
36.90 1.90
36.90 1.90
IFCA
41.50 1.33
41.31 z 1,99
41.43 a 0.76
41.46 a 0.57
40.54 4 1.22
40.53 a 0.88
40.46 199
40.26 1.73
FedRC
12,10 1,21
11.99 + 1.99
11.90 1.36
11.91 + 1,36
11.26 1.49
11.26 1.50
11.33 1.62
11.32 1.61
FedEM
11.98 + 1,28
11.68 + 1.21
11.90 1.37
11.89 1.37
11.26 + 1.49
11:26 + 1.49
11.33 1.62
11.32 1.61
FeSEM
19.07
38.91 + 0.99
39.21 + 0.85
40.26 a 0.56
39.59 + 0.99
40.40 + 0.58
41.07 + 0.91
40.17 = 1.52
FodDrift
0.13
39.02 + 0.99
38.89 0.79
38.89 2 0.79
38.09 = 1.46
38.09 + 1.46
37.91 #: L.85
37.52 1.75
CFL
39.10
39.00 + 0.96
38. 96.9622 0.79
38,96 = 0.79
38.15 1.56
38.15 + 1.56
37,90 L.89
37.90 1.89
PFedMe
39.58 1.01
NIA
40.95 + 0.38
NIA
41.62 + 1,50
N/A
43.
1.24
N/A
APFL
43.93 0.42
43.41 + 0.82
44.81 0.36 N/A
41.86 1.18
45.38 + 0.74
40.99 * 0.41
47.13 0.29
41.12 1.75
ATP
N/A
39,19 1,21
39,28 0.73
N/A
38.66 1.03
N/A
38.47 # 1,36
FLUX
37.40 + 1,28
36.53 # 1.81
38.81 0.98
37.58 1.24
39.70 # 1.03
37.59 # 1.72
38.65 2.62
37.00 1.23
FLUX-prior
38,15 1.51
37.99 # 1.50
38.68 # 1.27
37.32 1.74
38.95 + 1.85
36.00 + 1.53
41.16 1.1611.0 1.09
36.88 1.83
Table 38: Performance comparison across non-lID Levels 1-4 P(X on the CIFAR-10 10 dataset.
Known A.: Known Association.
non-IID Level
Level
Level
Level
Level 8
Algorithm
Known
Test Phase
Known
Test Phase
Known
Test Phase
Known A.
Test Phase
FodAvg
36,01 1.17
36.01 + 1,17
35.58 0.89
35,58 0.89
34.00 + 0.87
34.00 + 0.87
33.10 1.29
33.10 = 1.29
IPCA
40,02 0.72
39.61 + 1,11
38.99 : 1.66
18,74
38.08 + 1,35
37.41 0.73
$7.33 1.01
36,45 0.68
FodRC
10.90 0.95
I1.17
11,16 1.31
11.31 1.45
11.33 1.47
11.40 1.55
11:43 1.58
FedEM
10.92 1 0:99
10.90 0.96
11.8
11,16 1.31
11.31 1.46
11.33 1.48
1
11.43 1,58
FeSEM
41,07 1.04
39.48 1,51
40.37 1.32
38.44 0.75
40,00 1,01
37.42
1.16 40.21
36.39
0.94
FedDrift
37,22
37,22 1,21
36.59
36.17 + 0.75
34.62 1,01
34.05 1.76
34.02 0.78
33.16 1,93
CFL
37,20
37.20
36.35 0.79
36.35 0.79
34.62 + 1,05
34.62 1.05
33.39 1.33
33.39 1.33
pFedMe
44.64
N/A
45.44 1,00
NIA
45.02 0.76
N/A
APEL
0.11
47.25
+ 0.71
37.50 1.79
46.81 0.90
34,82
FLUX
41.14 1.17
36.09
2.50
41,43 1.81
37.45 1.92
39.38 2.26
37.50 2.90
37.65
3.63
33.83 3.81
FLUX-prior
41.65 0.77
36.59 2.48
41.46 0.77
38.77 a 1.17
41.66 a 0.67
37.67 1.02
42.35 0.82
36.59 2.08
Table 39: Performance comparison across non-IID Levels 5-8 of P(X on the CIFAR-10 dataset.
Known A.: Known Association.
non-IID Level
Level
Level
Level
Level 4
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phuse
FedAvg
=
42.01 + 1.00
37.40
42.48
IFCA
51.85
+ 3.24
1.07
FedRC
45.96 0.85
49.98 0.52
41.47 0.97
45.26 = 1.03
40.13 a 0.93
44.90 s 0.94
36.21 a 1.44
40,80 1.34
FedEM
46.60 a: 0.65
50.04 a 0.63
42.15 0.91
45.30 = 1.00
40.81 a 1.11
44.96
1.12
37.11 a
41.42
FeSEM
44.27 a 0.88
50.86 a 0.44
41.34 1.90
45.97 = 1.54
41.05 à: 1.63
46.85 a 1.12
37.27 a 1.74
43.08 1.78
FodDrift
4
41.16
2.48
42.42
36.28 = 2,10
42,00 + 0.83
35.08 # 1.55
38.62 1.99
28.14 1.20
CFL
47.79 0.42
50.47 2
43.02 1 0.87
45.61 = 0.87
42,41 + 0.88
46.42 = 0.80
38,30 1.08
42.40 0.82
PFedMe
3
N/A
34.41 # 0.45
N/A
34.53 + 2.95
N/A
34.43 1 4,81
N/A
APEL
49.64 0.33
33.65 a 0.32
47.73 + 0.68
34.72 = 0.37
47.83 a 1.05
36.63 a 0.77
44,28 a 4.50
36.50 1.74
ATP
N/A
0.43 $5.88
N/A
51.19 + 0.82
N/A
52.47 0.91
N/A
48.42 1.01
FLUX
42.85 1.33
43.29
3.59
40.46 #: 1.38
39.63 = 1.60
42.07 + 0.81
40.43 + 2.72
40.88 0.99
39.24 + 2.87
FLUX-prior
43.36 0.64
43.02 + 3.40
40.99 + 2.00
39.05 = 1.28
42.23 + 0.89
41.60 2.14
41.76 + 1.29
38.95 3.39
Table 40: Performance comparison across non-IlI Levels 1-4, summarizing all four types of heterogeneity (P(X), P(Y), P(Y]X), P(X[Y) P(XI)) the CIFAR-100 dataset. Known A.: Known
Association.
46

---

Level
Leve
Level
Algorithm
Known A.
Test Phase
Known A.
Test Phuse
Known A.
Test Phase
Known A.
Test Phase
FedAvg
29.86 4.39
33,76 + 5.05
26.24 = 0.62
30.32 = 0.70
23.34 0.87
28.09 a 0.90
20.36 2.69
25.63 3.10
IFCA
35.80 1 7.25
38.00 a 5.32
27.15 : 4.28
30.36 1 4.15
31.52 4,92
31.53 1 1.81
28.06 3.91
28.30 3.88
FedRC
31.85 a 1.76
34,28 a 1.52
26.19 1.57
30.17 a 1.47
25.82 0.87
29.57 a 0.98
22,92 0.81
28.08 = 1.05
FedEM
31.93 + 1.29
34.42 a 1.45
28.12 1.26
30.49 = 1.46
27.48
29.68 a 0.87
24.60 1.02
27.33 = 0.86
TeSEM
2.77
31.77 + 2.72
29.94 = 2.74
32.34 + 2.61
29.68
32.78 # 1.53
2
28.88 1.33
FedDrift
38.51 +.288
22,80 + 1.47
36.28 * 2.87
20.49 + 2.81
38.33 2.41
18.76 0,94
35.88 + 5.69
15,69 1.15
CFI.
31.85 2.26
35.61 + 2,56
27.31 + 0.73
31.28 + 0.81
24.80 0.40
29.81 * 0.38
21.47 3.02
27.12 3.48
PRelMe
37.38 0.78
NIA
37.54 * 0.76
NIA
38.67 0.50
N/A
38.34 0.95
N/A
APEL
42.59 0.98
38.03 + 0,73
41.15 0.66
38.67 0.45
40.78 0.39
39.71 + 0.49
39.95 0.57
40.32 = 0.52
ATP
NIA
43,99 2.33
N/A
38.14 1.15
NIA
38.34 + 0.59
N/A
35.63 = 1.86
3.57
Fraxx-prior
42.37
Table 41: Performance comparison across non-IID Levels 5-8, summarizing all four types of heterogeneity P(Y)), on the CIFAR-100 dataset. Known A.: Known Association.
non-IID Level
Level
Level
Level 3
Level 4
Algorithm
Known A.
Test Phase
Known A.
'est Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
31.99
31.99
+
38.48 + 1.34
30.03 2.19
IFCA
46.16
35.24 3,06
33.63
+
+ 1.47
30.69 1.22
30.69 1.22
TedRC
47.51 a 0.59
47.45 a 0.49
36.61 1.33
36.25 a 1.66
39.85 # 1.49
39.78 + 1.53
32.88 1.58
32.76 1.54
TedEM
47.34
0.70
36.71 1,30
36.33 1.48
39.85 + 1.83
39.71 + 1.83
32.89
32.82 1.73
FeSEM
42.94 0.35
46.39 + 0.56
34.24 2.42
34.24 + 2.42
38.31 + 1.07
39.35 + 1.36
0.73
32.79
1.63
FedDrift
4
31,58 1.37
36.34 4.91
22.65
1,58
39.60 0.78
28.65 + 1.61
32.51 + 3.72
22.61 1.18
CFL
0.50
44.75 0.50
32.38 + 1.34
32,38 + 1.34
38.17 + 0.94
38.17 + 0.94
30.01
0.85
30.01 = 0.85
pFedMe
= 0.06
N/A
29.85 0.23
N/A
31.06 #: 0.05
NIA
28.68
0.34
N/A
APEL
0.17
30.81 a 0.23
38.81 0.86
29.24 a 0.14
42.44 =: 0.59
30.74 a 0.08
37.04 0.73
28.70 = 0.43
ATP
NIA
49.52 0.31
N/A
36.84 # 1.21
N/A
43.41 + 1.16
N/A
34.17 1.41
FLUX
48.56 1 2.00
45.16 5.20
35.64
35.39 2.17
40.54 + 1.07
36.18
34.42 a 0.83
30.06 2.71
FLUX-prior
49.60 0.47
45.48 + 5.43
37.92 3,72
33,31 + 1.76
40.50 + 1,55
39.23 + 1.81
34. 34.80 2.25
30.14 3.40
Table 42: Performance comparison across non-IID Levels 1-4 P(X) on the CIFAR-100 dataset.
Known A.: Known Association.
non-11D Level
Level
Level 6
Level 7
Level 8
Algorithm
Known A.
Phase
Known A.
Test Phase
Known
Phase
Known A.
Test Phase
FedAvg
7,49 8.71
7.49 8.71
1,27 0.13
1,27 0.13
1.26 0.10
1,26 0.10
1.21 0.13
1.21 = 0.13
IFCA
25.80 13.11
24.35 9.08
5.43 6.26
5.65 6.58
19.64 8.22
15.51 2.96
9.71 6.18
9.83 6.26
FedRC
17.68 + 2.97
2.41
7,92 1.22
5.67 1.00
12.11 1.57
8.22 1.54
0.17
5.36 0.85
FedEM
17.50
+
8.08 1.39
5.82 1.07
11,95
7.94 1.38
+
5.46 0.78
FeSEM
4.66
18.46 4.559
13.02
9.89 2.76
18.62 4.03
13.92 1.83
12.24 1.19
7.70 0.82
FodDrift
34.61 2.33
11.90 1.55
23.45 3.07
8.35 0.28
30.33
10.18 1.02
1..17
7.66 1.45
CFL
2..32
4.39
4.00 0.11
6.35 0.20
6.35 0.20
3.63
3.63 0.27
PFedMe
27.98
N/A
25.77
N/A
APEL
27.50
0.32
26.02 0.22
0.28
25.73
+ 3.90
7,86 + 1.61
N/A
+ 0.54
N/A
8.00
FLUX
36,68 2.72
35,52 3.19
29.50 a 2.33
28.80 2.74
31.23 0.93
31.24 à 0.89
37.98 0.97
27,87 1.10
FLUX prior
36.61 2.82
34.56 3.33
32.15 1.73
30.51 4.04
31.22 0.60
32.27 0.51
28.43 0.91
28.33 1.09
Table 43: Performance comparison across non-IID Levels 5-8 of P(X) the CIFAR- 100 dataset.
Known A.: Known Association.
47

---

non-IID Level
Level
Level
Level
Level
Algurithm
Known A.
Text Phuse
Known
Test Phase
Known
Test Phuse
Known A.
Test Pluse
FodAvg
53.31 0.14
53.31 0.14
52.09 0.60
52.09 = 0.60
50.39 + 0.51
50.39 + 0.51
49.14 0.87
49.14 = 0.87
IFCA
53.76 a
53.76 a 0.56
2.09
2.95
39.83 : 6.00
42,60 1.05
44.11 2.74
FedRC
2
12.7220.55
0.29
50.31 a 0.33
47.55 0.25
47.50 0.26
44.52 1.75
44.48
FedEM
52.92 0.66
52.99 0.68
0.34
30,32
47.59 + 0.19
47.56 + 0.20
46.21 0.85
46.27:0:95
FeSEM
46.59 0.96
46.32 0.15
51.86 0.97
45,39 2.78
49.97 + 1.20
41.89 3.23
47.28 2.57
FodDrift
0.14 0.14
0.45
50,16 1.85
49,91 + 0,90
46.67 1.94
45.28 0.72
34.31 0.95
CFL
0.15
52.65 + 0.15
51.10
49.81 + 0.51
49.80 0.51
49.11
49.11 1.12
pFedMe
0.166
N/
31.93 0.45
N/A
28.29 5,88
NIA
26.90 9.59
N/A
APFI.
0.21
30.82 08
52.11 0.46
31,70
49.59 1.94
32.25 1.28
42.83 8.95
31.21 2.96
ATP
N/A
58.08 0.33
N/A
$7.70 0.58
57.24 0.41
N/A
57.38 0.98
FLUX
44.90 1,25
42.92 + 1.17
45.07 + 0.84
39.60 =: 0.29
44.85 + 0.76
37.48 a 2.37
46.03 +: 1.32
37.73 3.86
FLUX peior
45.01 + 1.11
41.27 1.07
44.80 0.98
39.51 0.29
44.82 0.71
37.83 2.24
46.00 1.19
37.80 3.83
Table 44: Performance comparison across non-llD Levels 1-4 of P(Y) on the CIFAR-100 dataset.
Known A.: Known Association.
non-IID Level
Level
Level
Level
Level
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phuse
FedAvg
49.23 0.66
49.23 0.66
48.18 1.07
48.18 = 1.07
45.94 1.49
45.94 + 1.49
42,70 5.32
4 5.32
IFCA
47.82 3.10
46,22 1.53
4.74
45.01 + 2.86
46.62 + 4.1
42.35 + 0.56
46.93 4.112
41.92 2.45
FedRC
46.25 0.98
44.98 + 2.09
44.72 2,31
44.44
44.11 + 0.71
45.99 1.53
45.95 1.58
FedEM
48.06 0.46
4 0.97
49.23 a: 2.05
45.46 2.26 2.26
51.28 a 1.32
44.72 a 0.56
52.31 a
43.61 1.24
FeSEM
46.46 2.57
49.37 0.95
43.38 + 3.85
44.93 = 3.53
46.11 a 2.21
46.62 = 1.74
44.62 2.29
45.84 2.00
FedDrin
48.30 0.93
32,16 1.87
48.77 + 3.04
30.10 4.66
49.57 2.19
26.50 1.25
51.81 + 3.56
22.06 1.02
CFI.
48.65 0.55
48.65 + 0.55
48.27 + 1,15
48.27 = 1.15
46.77 0.50
46:77 + 0.50
45.02 + 6.01
45.02 6.01
pFedMe
37.10
N/A
38.13 1.31
NIA
40.12 0.94
N/A
40.97 1,76
N/A
APEL
49.24 1.42
36.50 1.20
49.08 1.18
38.27 0,73
49.01 0.45
41.47 0.73
49.17 0.96
43.32 0.75
ATP
N/A
58.28 1.00
N/A
60.24 1.15
N/A
61.15 0.63
NIA
61.68 2.97
FLUX
$1.87 = 1.06
39.45
2.27
50.53 + 2.10
37.64 = 5.72
$3.69 + 1.02
43.22 a 4.69
56.33 0.92
48.31 3.34
FLUX peior
51.65 0.92
39.65 1,86
50.86 + 1.89
36.96 = 6.76
$4.73 0.85
43.58 5.04
56.32 1.12
48.36 3.73
Table 45: Performance comparison across non-lID Levels 5-8 8 of P(Y) on the CIFAR- 100 dataset.
Known A.: Known Association.
Level
Level
Level
Level 4
Algorithm
Known
Test Phase
Known
Test Phase
Known A.
Test Phase
Known
Test Phase
FedAvg
40.04 0.71
N/A
33,21
1.04
NIA
27.97
0.77
N/A
22.16 a 2.18
N/A
IFCA
NIA
1.44
4.56
FedRC
33.83 1.35
N/A
29.79 = 1.15
N/A
25.57
0.93
N/A
22.12 1.62
NIA
FedEM
36.32 0.65
N/A
32.44 0.92
N/A
28.21
1.16
N/A
24.03 + 1.43
N/A
F&SEM
37.29 + 1.42
N/A
34.73 1.27
N/A
30.33
1.27
N/A
26.53 + 0.96
N/A
FedDrift
37.73 0.40
N/A
34.76 1.30
N/A
3.55
N/A
27.01 1 0.69
N/A
35.25 0.884
NIA
30.38
1.09
NIA
PFedMe
30.94 0.07
31.15
N/A
0 0.33
30.87 + 0.13
APEL
43.34 4 0.36
N/A
0.54 42.58
NIA
39,84
0.33
N/A
37.29 4 0.33
N/A
ATP
43.21 0.73
N/A
18.10 0.91
N/A
32.98
1.21
N/A
28.20 4 1.24
N/A
FLUX
34.05 0.25
N/A
33.28 = 0.48
NIA
32.37 1 0.88
N/A
29.58 + 1.17
N/A
FLUX-prior
34.31 0.09
N/A
0.11
N/A
33.01
0
N/A
32.88
0.16
N/A
Table 46: Performance comparison across MID Levels 1-4 of P on the CIFAR-101 100 dataset.
Known A.: Known Association.
48

---

non-lID Level
Level
Level 6
Level
Level 8
Algorithm
Known
Test Phase
Test
Known A.
Test Phase
Known
Test Phase
FedAvg
18.16 0.86
N/A
14.01 0.24
N/A
9.06 0.74
N/A
4,53
0.09
N/A
IFCA
18.99 5.27
N/A
7.78
NAA
10.68 = 0.10
N/A
9.05 1.40
FedRC
20.72 1.63
N/A
11.22 2.01
NIA
9.54 0.42
N/A
4.72
N/A
FodEM
19,46 1.14
N/A
0.55
NAA
N/A
4.75
N/A
FeSEM
22.99 = 0.93
N/A
18.27 + 1.20
N/A
14.44 1.93
N/A
12.96
4.63
N/A
FedDrift
3.39
N/A
23.28 2.74
N/A
2.91
N/A
24.23 1.58
N/A
CFI.
0.95
N/A
15.40
N/A
9.78 0.46
N/A
4.51 0.27
N/A
PFedMe
30.67
30.95
N/A
APEL
33,04 1 0.21
N/A
32.60 2 0.24
NA
31.08 0.05
N/A
30.08
0.22
N/A
ATP
22.85 1.07
N/A
17.16 + 0.60
N/A
12.04 0.52
N/A
7.11 * 0.36
N/A
FLUX
31.01 1.07
N/A
28.39 4: 1.54
NIA
23.37 7,00
N/A
5.21
N/A
FLUX-prior
32.50 = 0.30
N/A
32.31 + 0.36
N/A
32.01 * 0.12
N/A
31.73 # 0.33
N/A
Table 47: Performance comparison across non-lII) Levels 5-8 of X(Y on the CIFAR-100 dataset.
Known A.: Known Association.
non-IID Level
Level
Level
Level
Level
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phuse
TedAvg
54.31 1.16
54.31 1,16
$3.96 + 0.62
53,96 = 0.62
51.21 # 1.15
51.21 s 1.15
48.27 0.53
48.27 0.53
IFCA
35.62 + 0.53
55.62 0.53
$2.78 3.13
51.77 = 1.75
50.60 0.56
50.35 = 0.40
49.31 + 0.35
46.82 + 0.26
FedRC
49.78 0.56
49.78
0.46
49.21 0.56
49.22 0.60
47.55 + 0.52
47.43 + 0.53
45.34 0.32
16
FodEM
49.84 0.50
49.76 0.48
19.22+00..
49.26 0.83
47.57 + 0.49
47.51 + 0.51
45.30 0.38
45.
0.27
FeSEM
50.26 a 0.29
52.65 a 0.48
50.07 + 0.95
51.80
$0.19 0.29
$1.22 a: 0.69
49.55 a 0.42
49.17 0.48
FedDrin
46.01 3.81
38.59
4.06
47.00 2.27
36.03 = 2.71
47.08 1.03
29.94 0.94
49.69 1.04
27.50 1.41
CFL
53.99 + 0.46
0.46 $3.99
53.34 0.37
0.37
53.34 = 0.37
51.28 0.87
51.28 0.87
48.07 + 0.16
48.07 0.16
pFedMe
39.66 0.63
N/A
44.71
0.68
NIA
48.20 0.31
N/
51.26 0.49
N/A
APEL
5 0.47
39.33
0.47
57,42 0.77
43.22 = 0,56
$9.46 0.45
46.90 0.37
59.95 0.41
49.58 0.30
ATP
N/A
60.04 0.60
N/A
59,02 0.50
NIA
56.75 0.99
N/A
53.52 0.28
FLUX
43.87 1.20
41.80 3.19
47.83 # 1.47
43.89 =: 1.69
50.51 a 0.40
47.65 a 2.36
53.50 0.34
49.92 1.61
FLUX peior
44.51 0.42
42.30 + 2.02
48.21 # 1.10
44.33 = 1.32
50.57 + 0.47
47.75 + 2.32
53.34 0.48
48. 90 2.80 2.86
Table 48: Performance comparison across non-lID Levels 1-4 of P(X on the CIFAR-100 dataset.
Known A.: Known Association.
Level
Level
Level
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
44.57 0.12
41.50 0.54
41.50 = 0.54
37.09 + 0.45
37.09 + 0.45
32.98 + 0.72
32.98 0.72
IFCA
50.67 0.94
43:43 0,33
50.12 1.91
40.42 1.42200.4
49.13 1.13:2222
36.74 = 0.87
46.56 1 1.97
33.14 0.21
FedRC
2.36
42.46 a 0.36
40.63 i 0.20
40.13 0.36
27.18
36.38 a 0.19
33.95 0.45
32.94 0.31
FedEM
42.69 0.29
42.46 a 0.38
40.60 0.15
0.15
40.20 = 0.34
37.16 2.11
36.39 0.25
33.92 0.50
32.92 0.31
FeSEM
46.60 0.48
45.47 0.53
45.11 1.63
42.19 = 0.65
39.56 0.92
37.80 0.82
40.67 +.433
33.12 0.82
FodDrift
5.08
24.36
0.77
49.63 + 2.61
23.01 = 1.40
50.25 + 3.13
19.60 0.22
44.877
17.35 0.92
CFL
44.96 0.18
44.96 0.18
41.58 0.79
41.58 = 0.79
36.30 0.35
36.30 0.36
3.370
32.70 0.37
pPedMe
43.77
N/A
$4.88 0.46
NIA
55,53 1 0.13
N/A
55.68 0.53
N/A
APFI.
60.56 0.55
50.20
59.91 0.43 N/A
51.73 0.14
$8.47 0.35 N/A
50.24 0.35
58.01 0.26
51.92 0.44
ATP
N/A
49.95 0.32
46.31 0.24
41.00
N/A
37.21
FLUX
55.27
54.30 1.53
56.17 + 0.63
51.32 1.30
56.53 + 0.06
52.80 3,08
56.62 0.45
$2.48 5.49
FLUX-prior
0.11
54.21 1.41
56.02 + 0.59
51.31 = 1.24
56.36 0.09
$1.26 3.53
56.73 0.27
52.66 5.51
Table 49; Performance comparison across non-llI Levels 5-8 of P(X on the CIFAR-10 100 dataset.
Known A.: Known Association.
49

---

limits augmentations to rotations (10", 90" 180%, 270" 1), with 6 classes subject to augmentation. Appendix F.1 describes more details about the non-IID levels setup.
To ensure sufficient training data per client, local datasets are duplicated based on the number of clients. For 5 clients, no duplication is applied (original size after partition). For 25 clients, datasets are duplicated once (size after partition x2). For 50 clients, duplication is applied twice (size after partition 4). For 100 clients, datasets are duplicated four times (size after partition x8).
Tables 51 to Table 55 provide detailed results for FLUX and baseline methods, evaluated across increasing numbers of clients and various distribution shift types and levels. We did not evaluate the performance of test phase on P(Y X) concept shift, as this problem is unsolvable without access to labels (refer to the Section 4.2.2 and Appendix C.4 for more details). We could not evaluate the performance of FedDrift and CFL under 100 clients due to prohibitive memory and computational costs (see Appendix B.3 for details). For the baselines that do not provide solution for real test phase, we weight all models by the number of clients in the cluster, and use the expectation that weights all model outputs as an estimation of the predicted labels.
Large-scale experiment with 1,200 clients. To further strengthen our evaluation and demonstrate
practical scalability, we extended the experiments to a real-world large-scale FL setting with 1,200 clients under partial participation. Specifically, we adopted participation rate of 0.2 (i.c., on average, 240 clients per round), which is both computationally practical and reflective of realistic deployments where device availability, network conditions, and battery constraints prevent full client participation. This setup follows the same protocol as Figure 5 for known association setting, but reports results separately for each distribution shift type. Table 50 reports results for FLUX and FedAvg. The findings confirm that FLUX not only scales effectively to 1,200 clients but also maintains strong robustness under extreme heterogeneity and limited participation. Notably, FLUX outperforms FedAvg by large margins-up up to 29.3 percentage points under the most challenging P(Y XX) shift-while exhibiting consistently low variance, highlighting its stability and scalability in realistic FL deployments.
Method
P(X)
P(V)
P(YIN)
P(XIV)
FodAvg
70.3 0.1
84.2 2.5
65.5 = 0.0
FLUX
96.1 0.0
98.2 + 0.2
94.8 0.3
96.3 + 0.1
Table 50: Large-scale scalability with 1,200 clients (partial participation rate 0.2).
# Clients
s Clients
25 Clients
Se Clients
100 Clients
Algorithm
Known
Text Phuse
Time
Known A.
Text Phaser
Time
Known
Tost Phase
Time
Known
Text Plase
Fediig
$9.09 a
4.443
75.34 =
6,662
56.30
70.75 a 2.45
7,471
65.7=
70.49 3.04
IFCA
80.1 = 6:08
03.33
e 4,27
6.684
76:1 s: 7.10
70,93 4.22
K666
76.24
2.99
70.32 e 3.01
10.965
74.45 e 3.95
09.32 + 2.10
10384
FaIRC
74.11
75.85
7,855
31.05=2.0 2.09
37.60 1.36
10.111
2.50
34.0 2.89
08.55
34, 34,90
FulEM
2.76
2.41
7.784
32.67
36.50
9,900
3.70
422
12.377
29.99 = 2.92
33.93 2.05
11,810
FiSEM
4.59
76.84 4.23
6,603
74.631.89 3.24
8,384
71.08
1.564
68.49
7
10:337
PedUnitt
0.20
55.62 3,00
8,072
70.37
62.11
14,102
82.17
3.28
21.91
16,481
N/A
NIA
N/A
CRI
86.88 1.65
88.71 x 1,18
7,690
63.92
1.76
64.44 2.45
14,211
67.16
2.04
71.87+233
16.195
N/A
N/A
N/A
JFulMe
9.11
N/A
5.619
89.21
0.81
X/A
6.23
89.06
0.77
NIA
7,760
89.06
0.63
9,081
APEL
0.30
2.23
7,210
89.41 = 0.76
2.66
K.92
0.67 59.35
22.10:2.16
9,313
71.83
10.313
ATP
NIA
88.83
5.862
NA
70.41 2.12
6.50
NIA
69.65
2.35
8,037
NAA
x
22051
95.35
5,436
3.165
2.16
6.738
3.55
84.72
7,962
3,04
2
9.263
FLUX pror
02275
98.06 : 0.25
5,753
90.20 = 0.47
90.28
1.67
5.796
90.36
0.40
89.24
1.42
7,960
90.50
0.33
89.06 2.08
9.252
Table 51: Performance comparison across the number of clients in 5, 25, 50, and 100, summarizing all four types of heterogeneity P(XX)))) Y)) on the MNIST dataset. Known A.: Known Association.
F.3 Robustness Under Mixed Distribution Shifts
To further assess the robustness of FLUX, we conducted additional experiments involving combina- tions of multiple distribution shifts occurring simultaneously. Specifically, we evaluated performance under three mixed-shift scenarios: (i) mixture of P(X) and P(Y) (Table 56); (ii) a mixture of
PYX and P(Y) (Table 57); and (iii) a mixture of P(Y|X) and P(Y) (Table 58), across four IID levels (see Table 9 for detailed setting of heterogeneity levels).
50

---

Bon-IID type
P(X)
P(Y)
POIX)
P(XIV)
Algorithu
Known A.
Text Phuse
Known A.
Text Phuse
Known
Test Phase
Known
Test Phase
FodAvg
85.59 = 0.35
85.59 + 0.35
92.90 1,00
92.90 1.00
70.50 2.62
N/A
88.77
1.25
88,77 1.25
IFCA
87.97 6.71
76.44 4.17
81.64 5.56
74,91 6.10
77.01 5.99
NIA
77.01
5.99
69.48 4.01
FedRC
78.29 1.26
74.30 2.09
85.66 + 2.82
83.83 2.90
67.44 3.42
NIA
67.44 + 3.42
65.64 3.52
FedEM
78.44 1.16
74.38 1.98
2
6.335
N/A
3.337
FeSEM
88.38 3.03
9.85
94.03 +
83.54 +
74.17 5.40
FedDrift
97.29 a 0.12
31.69 a: 0.51
98.68 a 0.28
40.56 a: 6.13
96.49 0.15
NA
0.21 97.46
65.89 1.04
CFL
85.14 + 0.78
25.09 + 0.24
92.86 # 1.25
26.41 + 3.54
70.58 2.58
NIA
88.
1.42
25.81 0.40
PFodMc
95.84 + 0.29
NIA
97.53 + 0.07
N/A
95.63 0.15
N/A
96.65 + 0.17
N/A
APFL
95.67 = 0.20
81.23 a 1.62
97.13 +: 0.18
89.31 + 3.28
95.17 0.49
N/A
96.66 +: 0,20
86.81 1.21
ATP
N/A
83.84 0.86
N/A
93.13 + 1.50
N/A
N/A
N/A
89.50 1.30
FLUX
97.59 0.21
97.63 0.18
98.69 0.56
97.84 2.23
85.71 2.54
N/A
92.04 3.08
90.57 1.44
FLUX-prior
97.67 + 0.15 97.67 + 0.15
98.79 + 0.32
98.80 + 0.32
96.75 0.14
N/A
97.71 + 0.24
97.71 0.24
Table 52: Performance comparison across 5 clients with all four types of heterogeneity (P(X), P(Y),
((X))) on the MNIST dataset.
non-TID type
P(X)
P(Y)
P(
P(X
Algorithm
Known. A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
67.53 1.61
67.53 a 1.61
78.56 a: 5.75
78.56 a 5.75
54.12 1.50
N/A
79.92 & 2.04
79.92 2.04
IFCA
82.49 + 3.07
65.23 + 2.34
79.68 + 13.31
68.95 a 6.36
61.00 3,30
N/A
80.88 + 2.06
78.61 + 2,71
FedRC
26.70 2.79
26.69 2.81
74.88 + 5.21
a.009
11.25 a: 0.93
NIA
11,35 0.00 0.00
11.35 0.00
FedEM
26.70 + 3.00
26.69 + 2.92
81,36 + 4.34
72.66 3.61
11.27 0.95
NIA
11,35 + 0.00
11.35 0.00
FeSEM
65.53 2.07
65.86 1.46
87.67 5.64
79.03 2.75
65.67
N/A
82.79 0.73 0.73
79.02 1.01
FedDrift
63,69 3.71
54.72 1,26
77.33 2.12
66.82 a 3,44
64.32 5.28
N/A
76.14 1.77
67.19 2.17
CFL
54.81 1,52
54.81 1,52
73.42 2.07
66.82 3.44
54.56 1.37
N/A
72.88 1.97
72.88 1.97
PFedMe
88.06 0.22
N/A
93.91
NIA
86.68
0.74
88.
N/A
APFL
94.10
8
86.63 1.31
ATP
N/A
57.91 1.65
N/A
82.18 2.36
N/A
N/A
N/A
71.14 2.28
FLUX
90.05 0.41
89.31 0.66
95.91 0.89
93.28 3.01
63.68 6.01
74.96 1.80
71.29
FLUX-prior
91.10
91.10
96.22
94.70 2.70
85.38 0.34
85.65
Table 53: Performance comparison across 25 clients with all four types of heterogeneity (P(X), P(Y), P(Y))) P(X(Y) on the MNIST dataset.
Bon-lID type
P(X)
P(Y)
P(Y
P(XJY)
Algoritium
Known
Test Phase
Known
Phase
Known
Test Phase
Test Phase
FedAvg
59.70 1 0.58
59.70 0.58
80447 3.94
80.47 1 3.94
52.53 1.08
N/A
72.08 1.45
72.08 1.45
IFCA
82.36 3.83
64.14 1.41
83.27 4.24
70.18 4.96
62.25 1.31
N/A
77.07 + 1.16
76.62 0.74
FedRC
26.36 1.85
26.35 1.85
63.25 3.66
11.07
12.76
FodEM
26.41 + 1.75
26.39 1.74
68.11
7.35
59.06 7.14
11.09 0.76
N/A
11.35
11.35
FeSEM
60.52 1.50
62.59 + 1.14
85.10 2.21
74.85 : 2.31
67.67 =
N/A
75.04 1.06
74.46
FedDrift
90.80 +: 2.77
40.04 + 2.45
96.48 + 0.41
46.98 + 2.72
65.48 5.30
NIA
75.74 + 2.65
68.69 2.48
CFL
61.23 2 0.65
61.23 + 0.65
81.41 + 3.42
81.41 # 3:42
53.04 + 1.03
N/A
72.97
+ 1.87
72,97 4 1.87
PFedMe
87.93 + 0.39
N/A
93.36 + 0.82
N/A
86.62 + 1.06
N/A
88.34
0.64
N/A
APEL
88.06 0.17
58.29 4: 1.29
93.93 a 0.90
84.42 a: 2.78
86.45 0.97
N/A
0.23 88.95
73.60 2.16
ATP
N/A
56.21 1.03
N/A
81.84 3.42
N/A
N/A
N/A
70.89 1.94
FLUX
90.83 + 0.34
90.13 + 0.75
96.04 = 0.42
93.69 1.52
69.40 6.76
N/A
70 2.14
70.35 2.19
FLUX prior
91.43 0.21
91.43 0.21
96.68 0.49
93.36 1.63
85.78 0.24
N/A
87.98 0.53
82.95 1.83
Table 54: Performance comparison across 50 clients with all four types of heterogeneity (P(X). on the MNIST dataset.
51

---

type
P(X)
P(Y)
P(Y
P(X
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
59.48 a 0.51
59.48 a 0.51
79.48 a 4.72
79.48 + 4.72
$2.02 a 1.16
NIA
72.52 + 2.26
72.52 2.26
IFCA
85.71 = 4.83
65,75 + 0.67
5.11
66,93 + 3.03
56.55 2.84
N/A
75.42
2.19 2.19
75,26 1.88
FedRC
26.39 = 1.69
26.39 1.69
65.29 3.44
65.17 3.34
11.6 0.78
N/A
11.35
0.00
11.35 0.00
FedEM
26.46
1.74
26.46 1.74
71.00 5.52
63,97 3.16
11,16 0.77
NIA
11.35
0.00
11.35 0.00
FeSEM
59.87
61.80 0.75
=
76.71
$
N/A
73.87
FedDrift
N/A
CHL
N/A
N/A
N/A
N/A
N/A
NAA
NA
N/A
PFedMe
87.99 a 0.08
N/A
93.34 = 0.99
N/A
86.64 0.66
N/A
88.26 & 0.40
N/A
APFL
$7.30 1.18 87.99 0.17
93.82 = 0.62
84.39 + 2.91
86.34 = 1.32
NIA
88,99 + 0.23
73,80 # 2.48
ATP
NIA
55.57 + D.15
NIA
81.18 + 3.50
N/A
N/A
N/A
71.19 + 2.04
FLUX
91.55 # 0.13
90.32 + 0.71
96.30 = 0.45
91.48 + 3.27
77.96 5.44
N/A
76.07 + 2.47
71.55 1.77
FLUX-prior
91.57 = 0.10
91.57 + 0.10
96.58 + 0.34
93.32 + 3.28
85.73 0.27
N/A
88.11 1..1100.4 0.49
82.28 1.47
Table 55: Performance comparison across 100 clients with all four types of heterogeneity (P(X), on the MNIST dataset.
Across all settings, FLUXX consistently maintains high accuracy, demonstrating strong robustness under compound shift conditions. contrast, existing CFL methods exhibit substantial performance degradation when faced with mixed-shift scenarios. As expected, personalized FL methods (c.g.. APFL, PFedMe) perform well under the known association condition, where test-time distributions match those where the models were fine-tuned on, achieving accuracy comparable to FLUX-prion However, their performance drops sharply on unseen clients, where labeled data is available for fine-tuning. ATP, which performs unsupervised test-time adaptation, remains relatively stable on unseen clients but fails to generalize across all distribution types, performing notably worse than FLUX. For example, under the P(X)+P(Y) mixture (Table 56), FLUX sustains test-phase accuracy of 95.9% at Level outperrrormig the best-performing methods by wide margin, e.g., ATP (85.4%), APFL. (80.6%), and CFL. (75.2%))and remains above 82.9% even at Level 6, where many baselines fall below 60%. Similar trends hold for the (Tables 57, 58), confirming FLUX's consistent robustness to compound shifts.
and P( mixtures
non-IID Level
Level 3
Level 4
Level
Level 6
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FedAvg
77.88
8.21
82.48 3,99
82.48 3,99
71.53 11.15
11.15
67.68 7.80
IFCA
86.35 7:45
69.4 15.54
95.78 0.44
81.85 2:31
90.31 1.39
69.30
12.37
85.97
66.00 = 11.00
FedRC
42.10 10.75
41.75 + 10.93
31.75 4.95
31,52 = 4.96
34.77 4.73
34.15
20.53 3.73
20.23 3.78
FedEM
66.02 4.55
42.09 + 10.75
30.69
59.60 21.72
35.07
a 6.03
50.71 4.33
19.96 3.69
FcSEM
78.67 4.83
9:51
75.93 + 11.99
66.20
20.17
78.87 7718
68.59
+ 13.27
+
59.38
11.38
FedDrift
97.20
37.51 5.97
95.23 0.75
38.62 a 5.32
95.39 1.82
34.82
10.98
37.00 4.55
CFL
9.556
75.18
81.92 = 4.88
81.92
67.54 13:22
67.54
13.22
72.49
5.84
72.49 5.884
pFedMe
97.26
N/A
96.08
0.35
95.58 1.84
95.04
APFL
0.39
80.58 : 6.58
96.25 = 0.20
80.67 = 3.92
95,82 + 1.29
69.30 = 13:45
94.71
0.995
63.18
17.98
ATP
N/A
85.39
3.64
NIA
77.30 + 15.52
NIA
68.12 + 12.86
NIA
72.07 + 9.56
1.74
95.91
2.06
6.08
86.78
4.10
87.16 2.27
81.73 4.64
83.44 3.80
+
FLUX-prior
97.33 a 0.72
97.33 a 0.72
96.13 a 0.45
96.13 a 0.45
95.77 a 1.66
95.77 + 1.66
95,17 1 0.49
94.97 0.49
Table 56: Performance comparison across mixture P(X) and P(Y) on the MNIST dataset. Known A.: Known Association.
F,4 Results on Real-world Datasets
To evaluate the real-world applicability of FLUX, we follow the setup described in Appendix B. and compare its performance against baseline methods on the CheXpert dataset across three levels of non-III heterogeneity (low, medium, high), using 20 clients. Since CheXpert multi-label classification task, we use macro-averaged ROC AUC as the evaluation metric instead of accuracy.
52

---

non-IID Level
3
Level
Level
Algorithm
Known
Test Phase
Known A.
Test Phase
Known
Test Phase
Known A.
Test Phase
FedAvg
69.61 5.58
69.61 5,58
78.94 7.09
80.06 4
68.93 5.49
68.93 0.71
IFCA
80.38 +
58.52 18.12
93.06
69.82 5.52
87.71 6.44
65.42 6.32
88.70 9.01
50.92
FodRC
59.14 12.52
55.28 8.47
6.316
64.02
+ 6.84
3.887
3.50
64.29 + 11.31
60.12 10.74
FodEM
38.79
57.00 6,93
17.97
63.29
+ 3.94
8,.40
59.86
3.42
85,96 5.73
58.91
10.21
FeSEM
83.02 4.49
61.57
3,60
88.46 2.56
66.
7.15 81.51
7
68.67 7.33
81.18
67.83
4,96
FedDrift
35.31 5.58
0.24
39.59
3.56 98.02
42.26
5.02
95,80 4.49
33.60
CFL
67.20 a: 4.42
67.20 a 4.42
76.88
1.42
76.88 1 1.42 78.76 a: 8.34
8.34 78.76
62.96 a 6.35
62.96 à: 6.35
pFedMe
98.35 à 0.22
NIA
98,96 a 0.31
N/A
98.97 + 0.40
NIA
99.34 a 0.07
NIA
APFL
99.03 0.17
64.09
6,38
0.03 98.77
75:87 4.28 98.66 + 0.32
75.33 6.18
98.91 0.20
69.58 + 5,40
ATP
N/A
70.27 7.99
N/A
N/A
82.10 7.02
N/A
63.84 9.93
FLUX
9..22
95.05
3.27
94.21 5.52
89.59
8.65 89.91 2.86
79.25
92.06 6.69
88.36 11.12
FLUX-prior
99.10 0.18
99.10 = 0.18
98.92 0.23 98.92 = 0.23 98.91 = 0.41
98.91 = 0.41
99.27 = 0.14
99.27 = 0.14
Table 57: Performance comparison across mixture of P(X and P(Y) on the MNIST dataset.
Known A.: Known Association.
non-IID Level
Level
Level 4
Level
Level 6
Algorithm
Known
Phase
Known A.
Test Phase
Known
Test Phase
Known
Test Phase
FedAvg
10.96
10.96
6.31 6.31
59.41 5.36
5:36
54.25 3.16
IFCA
95.96 0.71
75.32 + 12.02
83.89
56.60 13.64
83.64
7.72
46.16 8.738
3.56
FodRC
60.63 2 11.64
60.30 a 11.95
42.56 15.53
41.34 # 15.96
13.89
4.454
40.54
FedEM
73.77 + 5.83
60.42
10.92
59.59
40.04 + 16.06
$1.61 10,32
10.32
42.63
8.09
55.88 + 10.08
37.99 + 6.75
FoSEM
86.58 5.31
74.19
60.10 6.56
78.47
81.21 4.56
1.36
52.37 1 1.93
FodDrift
97.33 0.68
39.96 8.82
97.33
34.68 11.11
9
2
28.38 1251
CFL
$1.14 8.47
81.14
67.05
+
67 + 5.84
61.90
6.90 61.90
51.66
51.66 + 3.06
PFedMe
97.76 1: 0.30
N/A
97.95
0.43
NIA
97.54 a 0.21
N/A
97.39 #: 0.42
N/A
APFL
99.03 a: 0.17
81.75 a: 5.69
97.50 A: 0.48
66.61 à: 5.13
96.95 a 0.58
7.75 59.71
97.01 + 0.23
$1.26 a 4.52
ATP
N/A
81.94 10.74
N/A
70.22 6.05
N/A
60.23 00.33=3.000 3,90
N/A
53.47 + 3.08
FLUX
92.24 + 4.30
87.89 1 8.87
95.73
1.31
90.85 +: 3.48
92.98 a 3.90
87.13
94.17 4.64
91.25 à 8.77
FLUX prior
97.47 a: 0.49
97.47 0.49
98.00 0.39
98.00 0.39
97.47 0.25
97.47 + 0.25
97.05 0.54
97.05 0.54
Table 58: Performance comparison across mixture of P(Y and P(Y) on the MNIST dataset.
Known A.; Known Association.
The ATP baseline is omitted, as its unsupervised optimization procedure designed for multi-class classification and is not directly applicable to the multi-label setting.
As shown in Table 59, FLUX consistently and significantly outperforms baseline methods across all heterogeneity levels, in both the known association and test phase settings. In the known association setting, the best-performing baseline (APFL) degrades under increasing heterogeneity-drrpiin from 74.6 pp to 70.6 pp-whereas FLUX maintains strong and stable accuracy, achieving up to 80.3 pp under low heterogeneity and remaining above 79.2 pp even under the most challenging configuration. In the test phase, where clients are previously unseen and unlaheled, APFL's accuracy drops to 59.3 pp. while FLUX stays above 76.7 pp. Notably, most metric-based and parameter- based CFL baselines collapse to single global model during the training, demonstrating limited capacity capture real-world distribution shifts. By contrast, FLUX matches the performance of its oracle variant, FLUX-prior, which has access to the true number of clusters M, highlighting the discriminative strength of our learned descriptors and the effectiveness of the unsupervised clustering strategy. Finally, the difference between known association and test phase is negligible across most settings-except under high heterogeneity (M = 8), where a slight gap appears- appears-indicanl indicatiig that F1.UX can reliably assign previously unseen, unlabeled test-time clients to the appropriate clusters without supervision. These results highlight the practical viability of FLUX in federated medical imaging tasks, where substantial distribution shifts between training and deployment are common.
53

---

non-IID Level
Low
Medium
High
Algorithm
Known A.
Test Phase
Known A.
Test Phase
Known A.
Test Phase
FodAvg
$7.95 = 2.83
$7.95 = 2.83
55.81 = 0.00
55.81 = 0.69
54.53 = 0.78
54.53 0.78
IFCA
62.68 = 1.05
62.68 : 1.05
$1.79 0.48
$7.79 = 0.48
55.19 0.05
55.19
FedRC
62.84 1.06
62.84 1.06
58.08 = 0.48
56.78 = 0.48
55.42 + 0.06
55.31 = 0.06
FedEM
62.69 1,05
62.69 1.05
57.78 = 0.49
51.78
0.49
$5.19 0.06
55.19 0.06
FeSEM
62.54 1.15
62.51
1.16
58.04 0.17
57.48
0.45
56.45 0.26
54.96 0.06
FedDrift
67.46 1.08
67 07.46
1.08
60.60 1.28
60.60
1.28
56.87 0.13
56.87 0.13
CFL
62.67 1.06
62.67 1.06
57.79
0.48
57.79 0.48
55.20 0.05
55.20 0.05
pleedMe
68.75
68.56
N/A
0.60
0.70
71.64
0.77
59.23
FLUX
80.29 0.74
80.29 0.74
78.65
0.48
78.66
0.48
79.17 0.17
76.74 2.63
FLUX-prior
80.22
78.68 0.37
78.68 0.37
76.76 2.65
Table 59: Performance across three non-lID levels on CheXpert. Known A.: Known Association.
F55
Robustness to Partial Participation
We further evaluate FLUX under partial client participation, reflecting realistic FL. deployments where device availability, network conditions, or energy constraints prevent full participation in every round. Table 60 reports average accuracies on MNIST across all distribution-shift types, using the same experimental protocol as the scalability experiments Figure 5. Results are shown for different client participation rates, ranging from full participation (1.0) to as low as 0.2.
These results show that FLUX maintains strong performance even under significant dropout, consis- tently outperforming FedAvg. The most notable degradation occurs at a 0.2 participation rate, where only ~4 clients participate clustering. such limited participation can exclude entire distributions, naturally reducing performance. However, in large-scale FI., it is reasonable to assume that the number of participating clients per round exceeds the number of underlying clusters, ensuring sufficient distributional coverage. Importantly, unlike CFL/PFL baselines which generally assume full participation to isolate clustering effectiveness, FLUX extends naturally to partial participation. This opens the door to combining FLUX with existing client-selection strategies to further improve efficiency resource-constrained settings.
highly heterogeneous settings (e.g., M = 8 clusters),
Participation Rate
FLUX Accuracy (%)
FedAvg Accuracy (%)
1.0
91.3 + 0.4
76.7 + 2.0
0.8
91.2 1 0,3
75.9 : 1.4
0.6
75.0
+
72.3 1.0
0.2
82.0 + 2.2
70.8 4.00
Table 60: Accuracy of FLUX and FedAvg under different participation rates on MNIST.
F.6
Ablation Studies
F.6.1
Effeet of Client Data Size on Descriptor Quality
We further studied the robustness of descriptor estimation under varying local data sizes, simulating realistic FL scenarios where clients may contribute only few samples. Experiments were conducted on MNIST across four non-III levels, with client data sizes scaled to 4%, 8%, 16%, 32%, and 100% of the original. To isolate the effect of data size, other parameters were kept fixed. Table 61 summarizes results under the Known Association and Test Phase settings. As expected, overall accuracy decreases with smaller client datasets. Nevertheless, FLUX maintains substantially higher accuracy than FedAvg across all data sizes and heterogeneity levels. Even at only 4% of the data, FL.UX descriptors remain sufficiently informative to capture client heterogeneity, highlighting their suitability in practical FL deployments where clients often hold very limited data.
54

---

Methodi
Known Association
Test Phase
Low
Mid-low
Mid-high
High
Low
Mid-low
Mid-high
High
FLUX
$2.3
80.1 +
72.7
+
65.4 = 9.5
81.7
+ 2.1
76.9
2.3
5.7
7.2
FLUX 8%
83.7 2.3
80.5
1 3.9
70.3 1 8.3
66.9 1 9.7
81.7
1 1.9
77.7
2.1
66.4
7.1
62.2
8.6
FLUX 16%
87.3 + 1.8
84.6
2.4
77.8
6.0
70.0 5.7
84.0
+ 1.7
80.9
+ 1.5
73.3
5.2
66.0 = 5.8
FLUX
88.1 1.7
79.7
85.2
1.3
75.2
FLUX 100%
96.4 2.8
03.4
1.4 1.4
04.9 0.9
87.1 = 4.2
96.2
0.4
90.0
2.8
86.4 1.9
79.4
3.0
FodAvg
70.4
69.8 4.0
47.6
5.9
37.6
7.5
70.4
2.8
59.8
4.0
47.6
37.6
FedAvg
71.8
60.5
50.2
4.7
42.8
5.7
7188
2.4
60.5
3.1
5.72
42.8
5.7
FodAvg 16%
72.3 3.6
63.8
66.6
5.2
46.9
= 7.0
72.3
3.6
63.8
4.1
56.6
5.2
46.9
+ 7.0
FedAvg
2.7
54.4
72.1
2.7
3.3
62.0
3.3
2.9
FodAvg 100%
93.1
0.4
86.0
1.2
79.4
2.0
72.0
1.9
93.1
-
0.4
86.0
1.2
79.4
2.0
72.0 # 1.9
Table 61: Descriptor quality vs. client data size. Results on MNIST, averaged across four shift types and four non-IID levels, comparing performance under Known Association and Test Phase.
To further test robustness under extreme sparsity, we designed setting where clients hold only ~ ~1% the total data on average (Table 62). In this case, many clients contribute fewer than 1% of samples, leading very small local datasets. To enable convergence, we increased the number of clients to 20 and extended training to 200 rounds. Despite severe data scarcity, FLUX descriptors remain informative and significantly outperform FedAvg. This robustness arises from the design of our descriptors, which rely on low-order statistics (means and covariances) after dimensionality reduction, making them less sensitive to sparsity. While current implementation does not additional regularization, future work could further stabilize descriptor estimation using Bayesian shrinkage priors.
Known Association
Test Phase
Method
P(X)
P(Y)
P(YX))
P(X
P(X)
P(Y)
P(Y
P(X Y)
FodAvg
36.3
7.7
1.4
5.2
31.6 11.5
36.3
7.7
8.
N/A
31.6 11.5
FLUX
71.3 0.5
97.5
0.5
6.9
15.5 2 6.5
78.2 0.1
9,2
NA
42.0 9.3
Table 62: Descriptor robustness in extreme low-data regime (~1%). Results under Known Association and Test Phase.
F.6.2 Effect of Clustering Algorithm Choice
The primary contribution of FLUX lies in the design of informative, privacy-preserving descriptors that approximate the Wasserstein distance between client distributions. These descriptors enable meaningful unsupervised clustering while preserving privacy, unlike alternatives that rely direct metrics or gradient similarity. Consequently, the framework is agnostic to the specific clustering algorithm, requiring only that clustering operates solely on descriptors without relying prior knowledge such as the number of clusters (see Equation 6).
To validate this agnosticism, we conducted ablation studies comparing different clustering strategies. Table 63 reports results on MNIST, averaged across shift types, heterogeneity levels, and seeds. We compared Unsupervised k- k-Means, HDBSCAN, Agglomerative Clustering, and our density- -bsed method. Except for unsupervised k-Means, all algorithms achieve strong performance, confirming that FI.UX descriptors enable robust client grouping regardless of the specific clustering algorithm. Importantly, all methods converge to comparable accuracy in the test phase.
Method
Known Association (%)
Test Phase (%)
Unsupervised k-Means
86.8 14.6
93.5 3.8
HDBSCAN
90.5 7.7
92.9 2.6
Agglomerative
93.9 +: 3.2
94.2 0.8
FLUX (ours)
92.5 4 5.2
94.0 2.8
Table 63: Comparison of clustering algorithms on MNIST. Except for unsupervised k-Means, all
methods achieve strong accuracy, confirming that FLUX not tied to a specific clustering algorithm.
55

---

To further assess generality, we compared the two best- pert-perr methods (Agglomerative and ours) on CIFAR-100. Results in Table 64 show similar conclusions: both methods converge comparable accuracy, with differences within variance margins. This confirms that the robustness of F1.UX stems from its descriptors rather than the specific clustering algorithm employed.
Method
Known Association (%)
Test Phase (%)
Agglomerative FLUX (ours)
42.0 17.8
36.0 + 5.4
41.7 11.0
41.3 + 7,8
Table 64: Comparison of clustering algorithms on CIFAR-100. Both methods achieve similar performance, demonstrating that FL.XX remains robust across clustering strategies.
F.6.3 Effect of Test-Time Cluster Assignment Strategy
To evaluate the importance of our test-time assignment mechanism, we performed an ablation study where we replaced FLux's descriptor-basee assignment with a random cluster assigmment at inference. Following the same evaluation protocol as in Table I, we compared both strategies across all heterogeneity levels and distribution shift types on MNIST, FMNIST, CIFAR-10, and CIFAR-IO0
As summarized in Table 65, FLUX consistently outperforms random assignment by large mar- gin-up up 53 pp MNIST and 25 pp CIFAR-100. These results highlight the erucial role of our label-agnostic descriptors in reliably matching test clients to the most suitable cluster-specific models, even under severe non-IID conditions. Without this mechanism, performance degrades sharply, particularly in feature- and label-shift scenarios where random assignment fails to capture distributional similarity.
Dataset
Test-Assigament
P(X)
P(Y)
P(X
MNIST
FLUX
95.0 1.2
96.1
90.8
3.9
Random
41.9
69.2 17.2
73.2 13.7
FMNIST
FLUX
77.0
85.7
6.6
810
Rundom
14.5
61.1
63.1 11.1
CIFAR-I1 10
FLUX
33.3
46.2
1..
Random 20.45..1
318
3.7
CIFAR-100
FLUX
5.6
4.8
4.5
Random 17.0
15.9
3..1
Table 65: Ablation study test-time cluster assignment. Accuracy under different distribution shifts when using FLUX's descriptor-basee assignment versus random assignment. Results averaged across all heterogeneity levels.
F.6.4 Effect of Descriptor Configurations and Length
We performed an ablation study to evaluate the impact of different descriptor configurations in FLUX. Specifically, we tested various combinations of descriptors, including those approximating the marginal distribution P(X) (i.e., (uz x(() )) and label-wise descriptors capturing the conditional distribution P(Y]X) (i.e., { En dataset under four distinct distribution shifts, with non-lID levels fixed as described in Appendix F22 Furthermore, we analyzed the effect of the descriptor length 1, varying by scaling the output dimensionality of the reduction transformation (Equation 6) by factors of 1, 2, and 5.
(k)
>(A))) ) These experiments were conducted on the MNIST
Table 66 summarizes the average accuracy and standard deviation across all distribution shifts. The results in the known association setting underscore the clear utility of incorporating label-wise descriptors for P(Y) which enables FLUX to accurately identify clusters clients with similar distributions under label-conditional shifts. While the inclusion of standard deviation descriptors (a) for the latent distribution also leads improved accuracy, the performance gains are more modest. Interestingly, increasing the descriptor length (i.e., scaling it by or 5) did not yield improvements, likely due the reduced compactness of the client distribution representation, which complicates the clustering process. In the test phase, a flatter performance trend is observed. This behavior is primarily attributed two factors. First, the results for label-conditional shifts are not averaged during the test phase, test-time association without access to label data is inherently infeasible.
56

---

Second, label-wise descriptors cannot be used during testing as they rely on label information for computation. Consequently, FLUX relies solely on descriptors of P(X) in this phase, limiting the variability between different configurations.
Descriptors
Experiments
142
(kk)
(pm
(5)
(E
Length
Known Association
Test Phase
11
90.96 + 2.09
95.87 1.50
21
91.75 + 2.18
95.35 + 1.62
51
91.03 = 1.65
95.63 = 0.78
1[
92.62 # 1.19
95.21 = 1.39
21
90,32 2.75
95.51 143
51
89.83 2.55
94.72 1.33
11
93.07 0.75
95.55 2.09
21
91.58 + 2.55
95.02 1.14
51
90.25 + 2.48
94.92 # 1.26
11
93.86 + 1,35
95.64 = 1.63
21
91.82 # 1,34
96.06 = 1.11
51
90.76 # 3.57
94.95 = 2.13
Table 66: Results of the ablation study evaluating the impact of different descriptor configura- tions in FLUX under the Known Association and Test Phase settings. The configuration marked with indicates the setting used in this paper. The table shows average accuracy and standard devia- tion across four distribution shifts, highlighting the contribution of various descriptors approximating P(X) and PXY and analyzing the effect of descriptor length (!) on performance.
57

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
See Section 5.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
58

---

Justification:
See Appendix C.I.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: See the Experiment Settings section 5.1, and Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
59

---

Answer: [Yes]
Justification:
The data used is publicly available, and we provide the code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nips. wwtppcccpccccceecccccccccccccctt http://ggiieee/eeeeeaaeieeeeee
/Coo///oomissssseeeeee wwimiisssoooicyyyy. for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: See the Experiment Settings section 5.1 and Appendix B for the hyperparame- ters used, and Sections 5.2, 5.3 and Appendix for details the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification:
See figures and tables.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
60

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix B.3.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /ttpar//////....///eeee attps://errpppp.cppplll/////// ceweehhhhhhiieeeneneeneeaaee ines?
Answer: [Yes]
Justification:
We have thoroughly reviewed and adhered to the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See Section 5 and 7.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
61

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper does not address safeguards for the responsible release of high-risk data or models, as our research does not involve such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We the authors of the code used in this work. Information about other assets and the licenses of used models can be found in Appendices B.2 and B.3.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version the asset used and, possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi twwode..coo..... com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
62

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[Yes]
Justification:
See Appendices B.2, B.3 and the provided code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve research with user study.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
63

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: http:://eeuuupeeeeeee cct.cc///ooaaa http:/pps.ccccoofeeeeeeeeee////s for what should or should not be described.
64


Reviews:
[{"review_id": "8rZEFXzdFb", "reviewer": "Reviewer_L2wb", "summary": "This paper proposes a clustering-based federated learning framework, FLUX, which aims to address potential distribution shifts during both training and testing phases without relying on any prior knowledge. FLUX leverages a privacy-preserving descriptor extraction mechanism on clients to enable efficient unsupervised clustering and representation compression with low communication overhead. This design ensures both robustness and scalability of the global model, while enabling adaptive model selection for newly joined unlabeled clients during the testing phase. Extensive experiments on several benchmark datasets and real-world medical data demonstrate that FLUX outperforms 10 state-of-the-art baseline methods under varying degrees of data heterogeneity and task configurations.", "questions": "i. Has the performance of the descriptors been evaluated under varying levels of data noise?\n\nii.  How can the authors explain the subpar performance of the proposed method in certain cases?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n\ni. This paper proposes a label-free model assignment mechanism for the inference phase, addressing a key limitation of many clustered federated learning (CFL) methods that struggle to generalize effectively on the client side during testing.\n\nii. Comparisons were conducted across five datasets, eight levels of heterogeneity, four types of distribution shift, and both seen and unseen testing conditions.\n\niii. The paper follows a well-structured and readable format.\n\nWeaknesses\n\ni. The motivation of the paper is not sufficiently substantiated. Although it highlights that many existing methods rely on prior knowledge, it lacks a more systematic categorization or statistical evidence to convincingly support the claim that \"prior-free\" approaches offer broader generality.\n\nii. The experimental section provides insufficient explanation of FLUX-prior. Although this variant appears frequently in tables and figures, its definition and distinction from the main method are not systematically clarified.\n\niii. The description of the clustering method in the paper is overly brief; it only mentions \"density-adaptive clustering\" without specifying whether it refers to DBSCAN or another algorithm, and lacks details regarding the parameters used.\n\nⅣ. The applicability of hard versus soft clustering remains unclear. While the paper adopts a hard clustering strategy, it does not explicitly justify this choice or compare its effectiveness against soft assignment approaches.\n\nⅤ. The paper does not clearly articulate the underlying mechanism of the proposed approach, and it lacks illustrative examples that demonstrate how the method effectively tackles the identified challenges.\n\nⅥ. The text in Tables 1 and 2 is too small and may affect readability.", "was_revised": true, "final_justification": "It is recommended that the authors incorporate key clarifications and insights from the rebuttal into the revised manuscript, particularly those addressing reviewers' concerns. This will enhance the completeness and readability of the paper, and better align it with the expectations of top-tier conferences.", "rebuttal": "## Q1: Broader generality of \"prior-free\" approaches in FL\n\nWe have provided systematic evidence showing that prior knowledge is valuable when available. As shown in Table 1, FLUX-prior (with cluster number knowledge M) consistently outperforms FLUX across all datasets—achieving 95.7% vs. 94.0% on MNIST and 83.3% vs. 81.2% on FMNIST in the test phase—confirming the value of the prior.\n\nIn addition, the \"broader generality\" claim refers to applicability across deployment scenarios, not performance superiority. Most existing CFL methods (Table 2: IFCA, FeSEM, FedEM, FedRC) require knowing M, severely limiting their real-world deployment where this information is unavailable.\n\nAt last, **we have shown 'prior-free' FLUX offers broader generality: it achieves strong performance (within 1-2pp of the oracle) without requiring client heterogeneity patterns. This is essential in real-world FL, where data is decentralized across diverse entities and not existing access to the global data or the underlying distributions**. In such settings, assuming prior knowledge of M is unrealistic, and small trade-offs in accuracy are a necessary and worthwhile cost for ensuring deployability.\n\n## Q2: Extended details of FLUX-prior\n\nThank you for this feedback. We will add clearer explanation of FLUX-prior in the updated version. The key difference between FLUX and FLUX-prior lies in their clustering methodology:\n1. FLUX: An adaptive clustering based on DBSCAN that automatically determines the number of clusters without prior knowledge\n2. FLUX-prior: K-means clustering with the true number of clusters M as input (oracle knowledge)\n\nThe number of clusters is considered prior in FL settings, required by many baseline methods (e.g., IFCA, FedRC) as shown in Table 2. We introduced FLUX-prior to provide fair comparison with these prior-dependent baselines and to demonstrate the optimal performance achievable with our descriptor-based clustering approach using oracle information.\nAs expected, FLUX-prior consistently outperforms FLUX due to optimal clustering (e.g., 95.7% vs. 94.0% on MNIST, Table 1). However, **FLUX remains competitive with most baselines while requiring no prior knowledge—demonstrating that our automatic clustering approach achieves strong performance in realistic deployment scenarios where the number of client distributions is unknown**\n\n## Q3: Brief description of clustering method in FLUX\n\nWe thank the reviewer for pointing this out. In the revised version, we will include additional implementation details about the clustering procedure in Section 4.2.1. For how we allocate space for this update, please refer to our response to Reviewer Aotc, Q1.\nPlease note that **we kept the description of clustering method brief in the main text (while full details and parameter choices are provided in Appendix E.8), because the core contribution of FLUX lies in the design of informative, privacy-preserving descriptors that approximate the Wasserstein distance between client distributions—not in the clustering algorithm itself**. These descriptors enable meaningful unsupervised clustering while maintaining privacy, something that common alternatives based on metrics, or gradient similarity do not guarantee.\n\nCrucially, our formulation (Eq. 8) is agnostic to the choice of clustering method, requiring only that it operates solely on the descriptors without relying on external information such as M (i.e., unsupervised clustering). **To validate this, and as suggested by Reviewer LVoP, we conducted ablation studies with alternative clustering strategies**. These results show that, except for Unsupervised K-Means, all methods achieve strong performance, confirming that our descriptors enable robust client grouping regardless of the clustering algorithm. Notably, all methods converge to comparable accuracy in the test phase.\n|Method|Known Association (%)|Test Phase (%)|\n|-|-|-|\n|U. K-Means|86.8±14.6|93.5±3.8|\n|HDBSCAN|90.5±7.7|92.9±2.6|\n|Agglomerative|93.9±3.2|94.2±0.8|\n|Ours|92.5±5.2|94.0±2.8|\n\n## Q4: Justification of hard clustering in FLUX over soft clustering\n\nWe thank the reviewer for raising this point. We will clarify our motivation for using hard clustering in L111 of the revised manuscript. While soft-CFL approaches offer greater flexibility, we opted for hard clustering in FLUX for two main reasons:\n1. Computational and communication efficiency: Soft methods require clients to train and maintain multiple models locally and compute weighted combinations. This is impractical in cross-device FL, where the number of clusters M may be large and client devices are resource-constrained\n2. Limited test-time application: Soft approaches require learning a personalized weight vector $\\pi^k \\in \\mathbb{R}^M$ per client (see Eq. 11), which prohibits generalization to unseen clients at test time\n\n**For these reasons, hard clustering offers a more scalable and generalizable solution for real-world deployment within the FLUX framework.**\n\n## Q5: Mechanism articulation. Lack of illustrative examples\nWe appreciate the reviewer’s suggestion. **To clarify the underlying mechanism of FLUX, we now include a dedicated figure in the appendix that illustrates how the descriptor and clustering procedures operate in practice**. The figure presents examples on two datasets (MNIST and CIFAR-100) across three levels of heterogeneity (M=3,4,5). For visualization purposes, we project the client descriptors $d^{(k)} \\in \\mathbb{R}^L$ into a 2D space. These plots show that clients with similar data distributions are mapped close to each other in the descriptor space, enabling FLUX to cluster them correctly. The resulting clusters align with the latent distributional structure, confirming that the descriptors capture key distributional properties, even under complex heterogeneity. While we are unable to include figures in the rebuttal due to NeurIPS policy, we believe the descriptive content provided in the appendix improves the clarity of the FLUX mechanism.\n\n## Q6: Readability of Tables 1 and 2\n\nWe will increase the font size in Tables 1 and 2 to ensure better readability in the final version.\n\n## Q7: Performance under varying levels of data noise\n\nPlease note that we have already evaluated FLUX under multiple types of data noise and challenging conditions:\n1. **Distribution noise:** Our core experiments simulate realistic shifts across 8 heterogeneity levels for each of the 4 shift types—P(X) shifts via transformations, P(Y) shifts through class imbalances (from 10 to 3 classes), P(Y|X) shifts via label permutations (1-8 swapped classes), and P(X|Y) shifts through class-specific augmentations (1-8 classes under augmentation) (Section F.1, Table 7)\n2. **Real-world noise:** Our CheXpert medical imaging experiments involve 3 heterogeneity levels (low, medium, high) with naturally occurring noise from different imaging conditions, patient populations, and clinical settings—achieving up to 17.5pp improvement over baselines. Additionally, we have now included the Office-Home dataset, which contains four distinct domains, achieving a 2.1pp improvement\n3. **Mixed noise scenarios:** We tested 3 compound distribution shift combinations across 4 heterogeneity levels (Appendix F.4) where multiple noise sources occur simultaneously, with FLUX maintaining robust performance while baselines degrade significantly\n4. **Scalability noise:** We evaluated across 6 different client numbers (5, 10, 25, 50, and 100), where inherent FL aggregation noise increases, yet FLUX demonstrates superior stability compared to baselines\n5. **Sampling noise:** We evaluated descriptor robustness in the rebuttal results with 4 different client dataset sizes (4%, 8%, 16%, 32% of original), showing FLUX maintains high accuracy even with only 4% of original data\n\n## Q8: Explanation of subpar performance in certain cases\n\n**Explanation of the performance gap in (Fig11) on CIFAR-10 and CIFAR-100:**\nThe performance gap between FLUX and the best baseline has two reasons (We will also add those explanations to our updated manuscript):\n1. the only methods outperforming FLUX on CIFAR are Personalized FL (PFL) approaches. This is expected in scenarios involving P(Y|X) shifts (e.g., Tables 14–15), where each client may have different label preferences for the same input. In such cases, collaboration can be harmful, and methods fine-tuned on the same client distributions seen at test time (as in PFL) naturally have an advantage. However, these methods do not generalize to unseen clients (see Table 1, Test Phase), limiting their applicability in realistic deployments.\n2. insufficient model capacity may affect FLUX’s performance. On CIFAR-100, we observed that using a deeper backbone with a larger latent space (ResNet9 instead of LeNet5) reduces the performance gap, suggesting that limited model expressiveness can degrade the quality of the extracted descriptors, thus final accuracy.\n\n**Explanation of FLUX performance drop in Figure 5:** The performance drop is not specific to FLUX, but a general characteristic of FL under our experimental setup. Two key factors contribute to this trend (which we will also add to our updated manuscript):\n1. dataset size effects: In our experiment, client dataset sizes do not scale linearly with the number of clients—the actual size of local datasets decreases as we add more clients, leading to overall performance degradation regardless of the FL method used.\n2. inherent FL divergence: The nature of FL introduces higher divergence during model aggregation as the number of clients increases, which naturally degrades performance across all methods.\n\nAs shown in Figure 5, all baseline methods exhibit similar performance drops due to these same factors (Despite that personalization approaches can overfit their local dataset). However, FLUX still consistently maintains higher accuracy across all client numbers because the descriptor better captures client heterogeneity patterns."}, {"review_id": "oPpvX8HEkb", "reviewer": "Reviewer_c8ww", "summary": "The paper proposes a federated learning framework (FLUX) that leverages privacy-preserving client descriptors and unsupervised clustering to group clients, aiming to handle arbitrary distribution shifts without prior knowledge of cluster number or type. It supports\ntest-time adaptation for unseen, unlabeled clients and demonstrates empirical improvements over several baselines.", "questions": "- Could you clarify the significance of the Distribution Fidelity in Eq. 4? (difference between the euclidean distance and Wasserstein distance)? As the Wasserstein distance is computed between two distributions which is unknown, it is unclear how this metric can be used. In the appendix this is provided assuming Gaussian distribution which may or may not always hold.\n\n- Could the authors explain how does the design of the descriptors contribute to fulfilling the Distribution Fidelity requirement? This is not clear from the explanation in the appendix (908-926).\n\n- What are the key advantages of using DBSCAN over Hierarchical Clustering as in FL-HC[1]? Specifically, how does DBSCAN improve upon the hierarchical clustering in this context?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 1, "strengths_and_weaknesses": "Strengths:\n\n- The paper reports improved accuracy across multiple benchmarks and a real-world dataset, with overhead claimed to be comparable to FedAvg.\n\nWeaknesses:\n\n- The core idea—clustering clients for personalized federated learning—is not new and is wellestablished in the literature (e.g., MCFL [2] and FL-HC [1]). In FL-HC the server uses hierarchical clustering but the key idea is the same. The current paper uses DBSCAN where the radius epsilon is computed dynamically but the number off points per cluster is still the hyper-partameter. One-Shot Clustering is also explored in the context of unsupervised methods LADD [3].\n\n-  Lacks theoretical analysis of clustering such as impact of incorrectly clustered client and generalization.\n\n- The experiments on feature heterogeneity datasets such as DomainNet/Office-Home are missing.\n\n[1] Briggs, Christopher, Zhong Fan, and Peter Andras. \"Federated learning with hierarchical clustering of local updates to improve training on non-IID data.\" In 2020 international joint conference on neural networks (IJCNN), pp. 1-9. IEEE, 2020.\n\n[2] Long, Guodong, Ming Xie, Tao Shen, Tianyi Zhou, Xianzhi Wang, and Jing Jiang. \"Multi-center federated learning: clients clustering for better personalization.\" World Wide Web 26, no. 1 (2023): 481-500.\n\n[3] Shenaj, Donald, Eros Fanì, Marco Toldo, Debora Caldarola, Antonio Tavera, Umberto Michieli, Marco Ciccone, Pietro Zanuttigh, and Barbara Caputo. \"Learning across domains and devices: Style-driven source-free domain adaptation in clustered federated learning.\" In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 444-454. 2023.", "was_revised": true, "final_justification": "I thank the authors for their detailed rebuttal and subsequent post-rebuttal discussions. While these satisfied most of my original concerns, I still consider the novelty attributed solely to the design of client descriptors to be incremental within the broader context of clustering methods in federated learning. During the post-rebuttal discussion, I appreciate the thorough derivation provided under the Gaussianity assumption that shows how the descriptor distance approximates the Wasserstein distance. As most of my initial concerns have been addressed, I am happy to increase my rating.", "rebuttal": "## Q1: The novelty of FLUX compared to baselines such as FL-HC, MCFL, LADD.\n\nWe agree clustering in FL is established, and we thank the reviewer for providing the three important studies, which we will analyze in the Discussion section of the updated manuscript. However, please note that **our contribution differs fundamentally from existing clustered FL methods. While existing methods cluster based on model parameters [FL-HC, MCFL] or metrics (e.g., loss function, or frequency coefficients of the image [LADD]), FLUX clusters using novel distribution descriptors that directly capture client data distribution characteristics**. Moreover, FLUX is agnostic to unsupervised clustering algorithms, which enables FLUX to be equipped with suitable clustering approaches based on the use case.\n\nFurthermore, as shown in Table 2, existing CFL methods have critical limitations: \n1. metric-based approaches can miscluster clients with similar losses but different distributions. Similarly, considering LADD metric, clients with different conditional distribution P(Y|X) but same marginal distribution P(X), will be indistinguishable. \n2. parameter-based methods suffer from permutation invariance issues, i.e., models trained on same data but just different seeds may still have different parameters.\n   \nFLUX's descriptor-based clustering addresses these problems by approximating the 2-Wasserstein distance between distributions (Section 4.2.1) and capturing both marginal and conditional properties. This enables two additional key capabilities existing methods lack: (1) _handling all four distribution shift types simultaneously_ and (2) _test-time adaptation for unseen clients_. Our experiments show FLUX achieves up to 8 percentage points (pp) improvement over SOTA baselines while maintaining robust performance with unseen clients (Section 5.2).\n\n## Q2: Impact of incorrectly clustered client and generalization.\n\nWe agree on the importance of these points. Please note that **our empirical evaluation has already provided substantial evidence addressing these concerns**. The purpose of FLUX-prior, which uses oracle knowledge of the true number of clusters, is to serve as an upper bound for FLUX. Therefore, **the performance gap between FLUX and FLUX-prior (e.g., 94.0% vs. 95.7% on MNIST, Table 1) quantifies the impact of clustering errors**. Even at challenging high heterogeneity levels where clustering errors increase, FLUX still achieves 92.3% accuracy vs. 79.4% for the best baseline, remaining close to the oracle and showing resilience under imperfect clustering conditions.\n\nSimilarly, regarding generalization, we believe the evaluation setup provides extensive generalization validation: (1) Cross-dataset consistency across 5 datasets including real-world CheXpert; (2) Cross-shift robustness across all 4 distribution types and their combinations (Appendix F.4); (3) Scalability validation from 5 to 100 clients; and (4) Test-time generalization where FLUX maintains performance on completely unseen, unlabeled clients. **This empirical analysis provides compelling evidence for FLUX's robustness and generalization across diverse real-world scenarios**.\n\n## Q3: Missing feature heterogeneity datasets, like Office-Home.\n\nWe thank the reviewer for this valuable suggestion, which allowed us to further validate FLUX on a standard dataset exhibiting feature heterogeneity. **We conducted additional experiments on the OfficeHome dataset and results highlight FLUX’s effectiveness and resilience in real-world scenarios involving feature heterogeneity.** We followed the same experimental protocol used for CheXpert: each client was assigned randomly 200–600 samples drawn from a single domain, ensuring that all client data originates from one of the four domains.\n\nThe table below reports both Known Association and Test Phase accuracy. FLUX achieves strong performance on OfficeHome, outperforming all baselines in the test phase. As expected, only the personalized FL method APFL surpasses FLUX under the known association setting, as it evaluates models on the same client distributions used for fine-tuning. However, APFL suffers a noticeable performance drop on unseen test clients.\n\nNotably, FLUX matches its oracle counterpart (FLUX-prior) during the test phase, confirming its robustness even without prior knowledge of the number of domains (M = 4). Moreover, FLUX-prior remains competitive even with APFL while offering broader generalization capabilities. These results confirm that FLUX is a practical and effective solution for real-world FL deployments with feature heterogeneity.\n|Method|Known Ass. (%)|Test Phase (%)|\n|-|-|-|\n|FedAvg|37.1±0.9|37.1±0.9|\n|IFCA|32.8±8.1|29.6±7.5|\n|FedRC|22.2±2.8|22.2±2.8|\n|FedEM|28.4±2.3|28.8±1.8|\n|FeSEM|27.0±2.9|25.8±1.8|\n|FedDrift|39.5±4.0|34.6±4.1|\n|CFL|31.3±3.3|21.0±1.5|\n|pFedMe|30.9±2.3|N\\/A|\n|APFL|43.5±2.1|36.7±0.4|\n|ATP|N\\/A|37.9±0.9|\n|FLUX|39.2±0.2|39.2±0.3|\n|FLUX-prior|43.2±3.1|38.9±1.1|\n\n## Q4: Significance of the Distribution Fidelity in Eq. 4.\n\n**Eq. 4 defines a key requirement that descriptors must satisfy to be used within the FLUX pipeline—not an operational step**. It states that the Euclidean distance between descriptors should approximate a divergence (e.g., Wasserstein distance) between the underlying data distributions. Importantly, this condition is not used directly in practice and therefore does not require access to the true distributions $P(x^{(k_1)}, y^{(k_1)})$ and $P(x^{(k_2)}, y^{(k_2)})$. Instead, **it serves as a guiding principle**: descriptors should encode distributional differences such that clustering in descriptor space closely approximates clustering in data space (up to an error $\\xi$).\n\nFor our descriptor implementation, we derive in Appendix E.4 a closed-form expression under the Gaussian assumption of the latent space, which is a common choice since Gaussian distribution “makes the fewest assumptions and has maximum entropy” [Section 2.6.4 in ‘Probabilistic Machine Learning’, Di Kevin P. Murphy, 2022]. This enables analytical validation of distribution fidelity for our descriptors. However, our approach is not limited to this assumption: similar results can be verified numerically without any assumption on the underlying distribution (see Table 6 for empirical evaluations demonstrating this).\n\n\n## Q5:  How does the descriptor design ensure Distribution Fidelity?\n\nWe thank the reviewer for pointing this out. We will revise Appendix E.4 to clarify this concept. To satisfy the Distribution-Fidelity requirement R1, we design descriptors that approximate the distance between client data distributions in a compact form (see E.4). Each client first applies a client-invariant dimensionality reduction to each sample, then computes the first two moments (mean and covariance) over the reduced representations of all samples—both globally (to capture the marginal distribution) and per class (to capture the conditional distribution). The final descriptor concatenates these statistics. Because our client-invariant dimensionality reduction is designed to preserve distributional geometry, these latent-space moments serve as effective surrogates for the true data-space moments.\n\n**Crucially, as shown in Line 915, the 2-Wasserstein distance between Gaussian distributions depends solely on their means and covariances—the same quantities encoded in our descriptors. As a result, the Euclidean distance between descriptors is Lipschitz-equivalent to the 2-Wasserstein distance, up to a constant $\\xi$, ensuring Distribution-Fidelity.** \n\n\n## Q6: DBSCAN compared to Hierarchical Clustering in FL-HC.\n\nPlease note that the core contribution of FLUX lies in the design of informative, privacy-preserving descriptors that approximate the Wasserstein distance between client distributions—not in the clustering algorithm itself. These descriptors enable meaningful unsupervised clustering while preserving privacy, unlike many prior approaches (see Table 2). Our formulation (Eq. 8) is agnostic to the specific clustering method, requiring only that the clustering operates solely on the descriptors, without relying on external information such as the number of clusters (i.e., fully unsupervised).\n\n**To validate the agnosticism to clustering algorithms, we conducted an ablation study on the MNIST dataset using four clustering algorithms.** Results (averaged over all shift types, heterogeneity levels, and seeds as in Table 1) are shown below:\n|Method|Known Association (%)|Test Phase (%)|\n|-|-|-|\n|U. K-Means|86.8±14.6|93.5±3.8|\n|HDBSCAN|90.5±7.7|92.9±2.6|\n|Agglomerative|93.9±3.2|94.2±0.8|\n|Ours|92.5±5.2|94.0±2.8|\n\nThese results show that, except for Unsupervised K-Means, all methods achieve strong performance, confirming that our descriptors enable robust client grouping regardless of the clustering algorithm. Notably, all methods converge to comparable accuracy in the test phase.\n\nWe further compare the two best-performing methods (Agglomerative and Ours)—on CIFAR-100, with similar conclusions:\n|Method|Known Association (%)|Test Phase (%)|\n|-|-|-|\n|Agglomerative|42.0±17.8|36.0±5.4|\n|Ours|41.7±11.0|41.3±7.8|\n\n**This confirms that FLUX remains robust across clustering strategies, and our performance stems from the descriptors rather than the choice of algorithm.**"}, {"review_id": "xx5S93CoJL", "reviewer": "Reviewer_MC86", "summary": "This paper presents FLUX, a Clustered Federated Learning (CFL) framework designed to address data heterogeneity by leveraging client side data descriptors to form clusters in an unsupervised manner. FLUX can handle four data distribution shifts and train specialized models for each client group without needing to know the number of clusters in advance. Also, the framework supports assigning at the test time new and unlabeled clients to the most suitable model. Experiments show that Flux achieves good performance with minimal computational and communication overhead.", "questions": "(1) During training how were the different types of distribution shifts samples distributed across the clients?\n\n(2) Is there any investigation done to evaluate how the quality of descriptor degrades when amount of data available across each client changes?\n\n(3) If there are ties during cluster assignment, how is it resolved?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "***Strengths***:\n\n(1) The paper is presented very well and all the sections build upon one another, making it easy to read the papers\n\n(2) The integration of the different components to make FLUX functional is impressive. \n\n(3) Due attention has been paid to both the communication and computation cost and the tradeoffs are well justified\n\n***Weaknesses***:\n\n(1) It is not clear what is the novelty in the the proposed method. In a sense, I do not know if the prime contribution is formulating a new way to do clustering or is it more of a system the ties in already established techniques\n\n(2) In Figure (11) it is seen that for more challenging datasets such as CIFAR-10 and CIFAR-100, the performance of FLUX is not the best across different heterogeneity levels. Thus, it would be good to have some explanation on this observation.\n\n(3) In Figure 5, the drop in performance with device scaling for FLUX is not explained\n\n(4) It will be interesting to see an experiment (ablation study) that compares test time performance with random assignment of client to a cluster vs using the assignment protocol as mentioned in FLUX", "was_revised": true, "final_justification": "I thank the reviewers for providing responses to the questions. I will increase rating after seeing the responses.", "rebuttal": "## Q1: Novelty of FLUX\n\nOur novelty lies in the comprehensive approach to handle arbitrary distribution shifts in FL, not just clustering methodology. FLUX introduces three key innovations:\n1. **Novel distribution descriptors:** We propose privacy-preserving client-side descriptors that capture statistical characteristics of data distributions and facilitate client clustering process. Unlike existing methods that cluster on model parameters or loss values, our descriptors directly approximate the 2-Wasserstein distance between client distributions (Sec. 4.2.1, R1).\n2. **Unified framework for all shift types:** FLUX is the first method to simultaneously handle all four common shifts—P(X), P(Y), P(Y|X), and P(X|Y)—while supporting test-time adaptation for unseen, unlabeled clients. As shown in Table 2, no existing method satisfies all these requirement.\n3. **Automatic cluster discovery with test-time adaptation:** Our approach requires no prior knowledge of cluster numbers or distribution types, and enables previously unseen clients to benefit from appropriate cluster-specific models without labeled data.\n   \n**The contribution is a complete FL framework that integrates these novel components, achieving up to 8 percentage points (pp) improvement over SOTA baselines while maintaining FedAvg-level computational overhead**. This goes beyond tying established methods—it solves key limitations in existing CFL methods through principled design choices\n\n## Q2: Explanation of the performance gap in Fig. 11 on CIFAR-10/100\n\nWe thank the reviewer for the observation and will add a clarification around Line 270 in the revised manuscript. The performance gap between FLUX and the best baseline has two reasons:\n1. **the only methods outperforming FLUX on CIFAR10 and CIFAR100 are Personalized FL approaches**. This is expected in scenarios involving P(Y|X) shifts (e.g., Tables 14–15), where each client may have different label preferences for the same input. In such cases, collaboration can be harmful, and methods fine-tuned on the same client distributions seen at test time (as in PFL) naturally have an advantage. **However, they do not generalize to unseen clients, limiting their applicability in realistic deployments.**\n2. **insufficient model capacity may affect FLUX’s performance**. On CIFAR100, we observed that using a deeper backbone with a larger latent space (ResNet9 instead of LeNet5) reduces the performance gap, suggesting that limited model expressiveness can degrade the quality of the extracted descriptors, thus final accuracy. This may also impair the clustering process itself, as the descriptors might fail to capture subtle distributional differences between clients—potentially leading to under-clustering, where slightly distinct distributions are merged into a single cluster.\n\n## Q3: Explanation of FLUX performance drop in Fig. 5\n\n**The performance drop is not specific to FLUX, but a general characteristic of FL under our experimental setup**. Two key factors contribute to this trend:\n1. _dataset size effects:_ In our experiment, client dataset sizes do not scale linearly with the number of clients—the actual size of local datasets decreases as we add more clients, leading to overall performance degradation regardless of the method used.\n2. _inherent FL divergence:_ The nature of FL introduces higher divergence during model aggregation as the number of clients increases, which naturally degrades performance across all methods.\n\nAs shown in Fig. 5, all baselines exhibit similar performance drops due to these same factors (despite that personalization approaches can overfit their local dataset). However, FLUX demonstrates superior robustness compared to baselines—maintaining consistently higher accuracy across all client numbers. For example, in the test phase, FLUX sustains accuracy above 84% even with 100 clients. This robustness stems from our descriptor-based clustering approach, which better captures client heterogeneity patterns even when local data becomes limited.\n\n## Q4: Ablation study with test-time random cluster assignment\n\nWe thank the reviewer for this insightful suggestion. **To assess the importance of our test-time assignment strategy, we conducted an ablation study where we replaced our assignment mechanism with a random cluster assignment at test time.** Following the same protocol as in Tab. 1, we evaluated both methods across all 8 levels of heterogeneity for each type of shift on MNIST and CIFAR100. FLUX consistently outperforms random assignment by large margins—up to 53pp on MNIST and 25 on CIFAR100. This gap underscores the ability of our label-agnostic descriptors to match test clients to the most suitable cluster-specific models, even under severe shifts. We include these results in the Appendix.\n|Dataset|Test-Assignment|P(X)|P(Y)|P(X\\|Y)|\n|-|-|-|-|-|\n|MNIST|FLUX|95.0±1.2|96.1±1.5|90.8±3.9|\n|MNIST|FLUX-Random|41.9±11.4|69.2±17.2|73.2±13.7|\n|FMNIST|FLUX|77.0±2.9|85.7±6.6|81.0±1.4|\n|FMNIST|FLUX-Random|34.0±10.5|61.1±13.5|63.1±11.1|\n|CIFAR10|FLUX|33.3±2.5|46.2±7.8|36.7±1.3|\n|CIFAR10|FLUX-Random|20.4±5.1|31.8±5.4|32.3±2.7|\n|CIFAR100|FLUX|33.8±5.6|40.8±3.8|49.3±4.5|\n|CIFAR100|FLUX-Random|17.0±7.8|15.9±3.9|34.1±0.1|\n\nNote: We omit experiments on P(Y|X) shift, as test-time association in this setting is inherently unresolvable without accessing labels (see L223–224 and Sec. E.5).\n\n## Q5: Distribution shifts samples assignment during training\n\n**Each client is assigned a fixed-size dataset sampled from one specific distribution type, which remains consistent across all training epochs.** Specifically, we simulate the four shifts as follows:\n1. _P(X) shifts:_ Each client's images are applied with one specific color transformation and rotation.\n2. _P(Y) shifts:_ Each client only retains datapoints with certain labels.\n3. _P(Y|X) shifts:_ Clients randomly permute labels on their local data.\n4. _P(X|Y) shifts:_ Clients apply specific color and rotation transformations to designated classes.\n   \nTo ensure generality, we use widely recognized image augmentation approaches to introduce data heterogeneity (except for the real-world data CheXpert and OfficeHome, where we use the original images). We increase heterogeneity levels by expanding the number of augmentation choices available to each client. For details on data generation and distribution settings, please see Sec. F.1 and Tab. 7, which describe our non-IID construction.\n\n## Q6: Ablation study on quality of descriptor with variant available data size\n\nYes, **we conducted an ablation study on descriptor quality with varying client data sizes (4%, 8%, 16%, and 32% of original).** Experiments were conducted on MNIST across four non-IID levels, averaged over all four shift types. We adjusted other parameters accordingly such as rounds to isolate the impact of data size only on descriptor quality.\n\n**The results show FLUX maintains robust performance even with limited client data**. While overall accuracy decreases with smaller data sizes as expected, FLUX sustains high accuracy across different non-IID levels even with only 4% of the original data size. This shows that our distribution descriptors can effectively capture client heterogeneity patterns even when local datasets are small—a key property for real-world FL where clients often have limited data.\n\n_Ablation study on quality of descriptor_\n1. Known Association:\n|Method\\\\Data size|Low|Mid-low|Mid-high|High|\n|-|-|-|-|-|\n|FLUX 4%|82.3±3.6|80.1±4.3|72.7±7.3|65.4±9.5|\n|FLUX 8%|83.7±2.3|80.5±3.9|70.3±8.3|66.9±9.7|\n|FLUX 16%|87.3±1.8|84.6±2.4|77.8±6.0|70.0±5.7|\n|FLUX 32%|88.1±1.7|85.0±1.7|79.7±4.8|66.8±7.3|\n|FLUX 100%|96.4±2.8|93.4±1.4|94.9±0.9|87.1±4.2|\n|Fedavg 4%|70.4±2.8|59.8±4.0|47.6±5.9|37.6±7.5|\n|Fedavg 8%|71.8±2.4|60.5±3.1|50.2±4.7|42.8±5.7|\n|Fedavg 16%|72.3±3.6|63.8±4.1|55.6±5.2|46.9±7.0|\n|Fedavg 32%|72.1±2.7|64.8±3.3|62.0±3.3|54.4±2.9|\n|Fedavg 100%|93.1±0.4|86.0±1.2|79.4±2.0|72.0±1.9|\n2. Test Phase:\n|Method\\\\Data size|Low|Mid-low|Mid-high|High|\n|-|-|-|-|-|\n|FLUX 4%|81.7±2.1|76.9±2.3|68.5±5.7|60.8±7.2|\n|FLUX 8%|81.7±1.9|77.7±2.1|66.4±7.1|62.2±8.6|\n|FLUX 16%|84.0±1.7|80.9±1.5|73.3±5.2|66.0±5.8|\n|FLUX 32%|85.2±1.9|81.5±1.3|75.2±4.7|62.6±6.4|\n|FLUX 100%|96.2±0.4|90.0±2.8|86.4±1.9|79.4±3.0|\n|Fedavg 4%|70.4±2.8|59.8±4.0|47.6±5.9|37.6±7.5|\n|Fedavg 8%|71.8±2.4|60.5±3.1|50.2±4.7|42.8±5.7|\n|Fedavg 16%|72.3±3.6|63.8±4.1|55.6±5.2|46.9±7.0|\n|Fedavg 32%|72.1±2.7|64.8±3.3|62.0±3.3|54.4±2.9|\n|Fedavg 100%|93.1±0.4|86.0±1.2|79.4±2.0|72.0±1.9|\n\n## Q7: Solution to ties during cluster assignment\n\nThank you for this question. **Exact ties during cluster assignment in FLUX are extremely rare in practice. Given that our descriptors are continuous-valued statistical moments computed from client data distributions, and considering the natural variation in real datasets, the probability of two clients having identical descriptor values is negligible.**\n\nWe do encounter situations where descriptors are close and near clustering boundaries. This typically occurs when descriptors are over-compressed or when the model's latent space representation is insufficient—which contributes to FLUX's performance drops at very high heterogeneity levels. When clustering uncertainty occurs, FLUX assigns clients to the nearest available cluster based on Euclidean distance (Eq. 9, Sec. 4.2.2). Even with potentially suboptimal assignments, FLUX demonstrates resilience by assigning clients to the second-closest cluster, which typically shares similar distributional characteristics. This robustness is evidenced by our comprehensive experiments showing that FLUX maintains high accuracy across all heterogeneity levels—consistently outperforming baselines and remaining close to the oracle FLUX-prior, even when perfect clustering becomes challenging.\n\nThe statistical nature of our descriptors naturally reduces tie probability, as moment-based representations capture continuous distributional characteristics rather than discrete values."}, {"review_id": "XS6VKzXuLd", "reviewer": "Reviewer_Aotc", "summary": "In Federated Learning (FL), dealing with data that is non-independent and identically distributed (non-IID) across clients can make generating a global model that accurately fits the overall distribution challenging. To tackle this issue, one approach is to shift the paradigm by grouping clients based on their local data distributions. This method is referred to as Clustered Federated Learning (CFL). However, many existing CFL approaches tend to focus only on specific types of distribution shifts, which limits their ability to provide a comprehensive understanding of how their methods perform under feature distribution shifts, label distribution shifts, and concept shifts.\n\nMoreover, most of these methods make unrealistic assumptions, such as knowing the number of clusters in advance or being able to assign test clients to existing clusters beforehand. To address these challenges, this paper introduces a novel cluster-based approach called Federated Learning with Scalable Unsupervised Clustering and Test-Time Adaptation (FLUX). \n\nFLUX assigns a descriptor to each client, which is constructed using class-conditional moments and can be enhanced with differential privacy to improve privacy guarantees without significantly sacrificing performance. These descriptors are then utilized to cluster the clients using a new technique inspired by DBSCAN, after which each cluster undergoes FL training. Extensive experiments demonstrate the effectiveness of this method.", "questions": "1) This paper is based on the assumption that traditional FL methods rely on the idea that clients' data is IID, which is often not the case. Indeed, this is a key distinction between Distributed Learning (DL) and FL, as noted in [2], Table 1. Furthermore, several studies have addressed this specific issue within FL, including works already cited by the paper (e.g., SCAFFOLD). Could the authors provide a convincing justification for this claim?\n\n2) How is Eq. 2 derived?\n\n3) How does FLUX relate to the method proposed in [3]?\n\n[2] Kairouz, Peter, et al. \"Advances and open problems in federated learning.\" Foundations and trends® in machine learning 14.1–2 (2021): 1-210.\n\n[3] Licciardi, A., et al. \"Interaction-Aware Gaussian Weighting for Clustered Federated Learning.\" Forty-second International Conference on Machine Learning (ICML), 2025.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n1) The effectiveness of the method is demonstrated through numerous experiments conducted on five datasets, comparing it with several baseline approaches.\n\n2) FLUX does not require prior knowledge of the exact number of clusters.\n\n3) The paper effectively addresses four types of distribution shifts in the experiments by investigating each one separately.\n\n4) The overall description of the method is convincing, and the motivations are generally well discussed, although there are some exceptions (see the weaknesses and questions).\n\n### Weaknesses\n\n1) The main paper provides a high-level overview of the method, while the specific details on how the individual components have been implemented are found in the Appendix. Although this choice is likely due to page limits, the authors might consider including additional details to clarify the method in the main paper while removing repetitions (e.g., some symbols have been introduced multiple times, or the entire background section could be simplified). Currently, the description can be somewhat confusing, making it difficult to understand the ideas behind its construction.\n\n2) FLUX requires all the clients to be available in every round, which could be an unrealistic assumption, especially in cross-device FL scenarios.\n\n3) Although the paper already provides a discussion on the efficacy of FLUX with various numbers of participating clients, experiments with a large number of clients (>1000) or more real-world problems (e.g., images with higher resolution) are missing. For instance, see the datasets proposed in [1].\n\n[1] Hsu, Tzu-Ming Harry, Hang Qi, and Matthew Brown. \"Federated visual classification with real-world data distribution.\" Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16. Springer International Publishing, 2020.", "was_revised": true, "final_justification": "Based on the current state of the rebuttal and the answers provided by the authors, I still lean towards acceptance.", "rebuttal": "## Q1: Including more technical details in the Method section\n\nWe thank the reviewer for this suggestion. In the revised version, we will improve clarity and reduce repetitions by:\n1. Removing the repeated definition of variables in the Problem Statement paragraph, already introduced in the Background\n2. Simplifying and shortening L68–72 and L95–98 in the Background, and remove the non-necessary sentence in L138–141.\n   \n**With the recovered space and the extra page, we will integrate technical details currently in E.5 (e.g., L889, 894, and 899) into the main text.** These additions will enrich the Practical Implementation part of the Descriptor Extraction and Unsupervised Clustering paragraphs.\n\n## Q2: Assumption on all clients’ participation\n\nFLUX **does not require all clients to be available in every round and makes no assumptions about client participation**. In our experiments, we assume full client participation only to ensure fair comparison with baselines, which mostly require this assumption.\n\nThis is a standard procedure in FL literature for controlled evaluation, not a fundamental requirement of FLUX. FLUX only requires that a newly joint client submits its descriptors before being assigned to the appropriate cluster (the clustering process can also be repeated), then it can freely leave and rejoin training at any epoch within its designated cluster.\n\nThe clustering-based design of FLUX actually makes it more robust to intermittent participation than traditional FL—when clients rejoin, they immediately benefit from their cluster-specific model rather than a potentially mismatched global model. This flexibility is particularly valuable in real-world deployments where client availability varies significantly. \n\n## Q3: Large scale experiment and real-world experiments\n\nWe appreciate this suggestion, though experiments with >1000 clients are practically infeasible and beyond current FL research standards. Current FL research typically evaluates with 10-100 clients, including ATP (NeurIPS 2023), FedDrift (AISTATS 2023), and established methods like IFCA use similar or lower scales. **We deliberately followed these established experimental protocols to ensure reproducible and comparable results.**\n\nOur evaluation up to 100 clients follows the experimental scope of existing baselines and recent FL literature—in fact, several methods (FedDrift, CFL) could not even complete experiments with 100 clients due to prohibitive memory and computational requirements (Sec. E.2), despite access to 512 GB RAM and 4 GPUs with 49 GB VRAM each. Scaling to >1000 clients would make fair baseline comparison impossible, as most existing methods cannot handle such scales. \nMore importantly, **our theoretical analysis and experimental trends clearly demonstrate FLUX's scalability advantages**. As shown in Fig. 5 and Tab. C, FLUX maintains computational complexity comparable to FedAvg (O(Kp) vs. baselines requiring O(K²p) or O(MKp)), with communication overhead of only L/p ≤ 3.5×10⁻³. These theoretical guarantees, combined with empirical trends up to 100 clients, provide strong evidence for scalability.\n\n**We believe our evaluation scope—100 clients across 6 datasets (with OfficeHome added during the rebuttal phase) and 10 baselines—represents one of the most comprehensive CFL evaluations in recent literature. To further demonstrate scalability, we pushed our system to its computational limits using the most efficient methods (FedAvg and FLUX), successfully scaling to 400 clients (Table below)**. On MNIST, FLUX achieved up to 98.9% accuracy in the test phase under P(Y) shifts, outperforming FedAvg by 3.7 percentage points, while maintaining strong performance across all other shift types. Even under challenging P(Y|X) conditions, where no test-time association is possible, FLUX reached 95.6% in the known-association setting vs. 58.0% for FedAvg. These results highlight FLUX’s ability to scale efficiently and maintain high accuracy even in large-scale, non-IID settings.\n|Shift Type|Known A. (Ours)|Known A. (FedAvg)|Test Phase (Ours)|Test Phase (FedAvg)|\n|-|-|-|-|-|\n|P(X)|98.3±0.1|83.5±0.1|97.2±0.4|83.5±0.1|\n|P(Y)|99.3±0.0|95.2±0.8|98.9±0.2|95.2±0.8|\n|P(Y\\|X)|95.6±1.9|58.0±0.2|N/A|N/A|\n|P(X\\|Y)|92.6±1.4|90.1±1.1|88.7±5.9|90.1±1.1|\n\n## Q4: Justification for claim on papers of IID assumption\n\nWe agree that the non-IID problem in FL is well-recognized. However, our foundational claim remains valid for three key reasons:\n1. **Many technical FL papers still ignore non-IID issues.** Papers focusing on FL privacy enhancement (e.g. differential privacy) and communication efficiency (e.g. pruning, compression) typically assume IID data to simplify analysis while concentrating on their primary contributions. According to our counting on recent publications in three digital libraries: IEEE-X, SpringerLink and ACM-DL, within the year 2024, roughly 60% papers in these domains operates under IID assumptions, indicating that current SOTA approaches may not work in real-world non-IID settings.\n2. **FL application works frequently assume IID cases.** Papers of FL application also may ignore the data issue. For example, we count that, according to a recent survey on FL in HAR applications (Grataloup A. A systematic survey on the application of federated learning in mental state detection and human activity recognition. Front Digit Health. 2024), only about 40% of papers consider non-IID as an issue in their proposed systems. This demonstrates that many FL researchers still overlook non-IID challenges during deployment, reinforcing the practical relevance of our work.\n3. **Existing non-IID FL methods have significant limitations.** While methods like SCAFFOLD address non-IID data, they fail to provide comprehensive solutions. As shown in Tab. 2, FLUX uniquely satisfies all critical requirements: handling all four non-IID types simultaneously, requiring no prior knowledge, supporting test-time adaptation, and maintaining low computational costs. SCAFFOLD, for instance, lacks test-time adaptation capabilities and doesn't specifically address the four distinct shift types that FLUX handles.\n   \nTherefore, while non-IID awareness exists, practical solutions that comprehensively address arbitrary distribution shifts remain limited, justifying FLUX's contribution.\n\n## Q5: Derivation of Eq. 2\n\nWe thank the reviewer for the comment and will include the following paragraph in A.1.1 for clarification. \nEq. 2 is written in a general form that subsumes both soft- and hard-CFL (separate, standard formulations for each appear in A.1.1.). Because hard-CFL is simply the special case of soft-CFL where each $c^{(k)}$ is a one-hot vector, we start from the general soft-CFL likelihood (e.g., FedRC).\n\nIn soft-CFL, each client’s data are assumed to be generated from a mixture of $M$ client-level distributions $P(Y \\mid X; \\theta_m)$ with mixing weights $c^{(k)} \\in [0,1]^M$ such that $\\sum_{m=1}^M c^{(k)}_m = 1$. For client $k$ with dataset $(x^{(k)}, y^{(k)})$, the conditional log-likelihood is (Eq. 1 in [9]):\n\n$\\log P(y^{(k)} \\mid x^{(k)}; \\Theta, c^{(k)}) = \\sum_{(x,y) \\in (x^{(k)}, y^{(k)})} \\log \\left[ \\sum_{m=1}^M c^{(k)}_m P(y \\mid x; \\theta_m) \\right]$,\n\nwhere $\\Theta = {\\theta_1, \\dots, \\theta_M}$.\nAs in mixture-model derivations, we introduce the standard Jensen (EM) lower bound to replace the inner log-sum with a weighted sum of logs:\n\n$\\log \\left[ \\sum_{m=1}^M c^{(k)}_m P(y \\mid x; \\theta_m) \\right] \\ge$\n\n$\\sum_{m=1}^M c^{(k)}_m \\log P(y \\mid x; \\theta_m)$\n\nFinally, summing this bound over all samples, clients, and clusters yields the surrogate objective presented in Eq. 2.\n\n## Q6: Relations between FLUX and the suggested work by Licciardi, A., et al. (ICML 2025)\n\nWe thank the reviewer to provide the opportunity for us to compare FLUX with FedGWC, which we will analyze in the final version of our manuscript (it was not included in our original submission because it was published very close to the submission deadline, making it unavailable during our initial literature review and experimental design phase).\n\nBoth FLUX and FedGWC work on CFL to address data heterogeneity across clients by grouping clients with similar data distributions.\n**The key difference is that FedGWC uses empirical loss patterns and Gaussian weighting mechanisms to identify client clusters based on loss landscape similarities, while FLUX employs privacy-preserving descriptor extraction that captures statistical characteristics of client data distributions to enable unsupervised clustering across four distinct types of shifts.** \n\nFedGWC provides elegant mathematical framework with strong theoretical foundations, including convergence proofs for Gaussian weights as unbiased estimators and the introduction of the Wasserstein Adjusted Score for clustering evaluation in FL. However, it may need further validation on different shift types, as metric-based approaches (e.g. using the loss) can miscluster clients with similar losses but different distributions.\n\nFLUX complements FedGWC's contributions while addressing several additional challenges: \n1. _Broader distribution shift scope_—FLUX explicitly handles all four fundamental types of shifts (P(X), P(Y), P(Y|X), P(X|Y)) while FedGWC primarily focuses on P(Y) and P(X|Y) shifts\n2. _Enhanced test-time capabilities_—FLUX enables previously unseen and unlabeled clients to automatically find appropriate cluster models without prior training participation, extending beyond FedGWC's training-time clustering\n3. _Extensive empirical validation_—FLUX demonstrates effectiveness across six datasets including real-world CheXpert and OfficeHome with comprehensive evaluation across different heterogeneity scenarios\n4. _Alternative clustering approach_—FLUX's descriptor-based method provides a complementary perspective to FedGWC's loss-based approach, potentially avoiding scenarios where clients with similar empirical risks have divergent distributions"}, {"review_id": "nRmcdtyns1", "reviewer": "Reviewer_LVoP", "summary": "The paper introduces a method that efficiently handles various data distribution shifts during both training and test phases. The proposed method lies in its descriptor-driven clustering approach that extracts client-side descriptors through privacy-preserving feature extraction, followed by an unsupervised clustering algorithm to group clients with similar data distributions. The method was evaluated across multiple datasets, demonstrating good performance under various levels of heterogeneity and distribution shifts.", "questions": "None.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "This paper presents a well-structured and logically coherent work with good empirical performance. The authors provide a detailed report and experimental results on various heterogeneity and distribution shifts. However, several aspects require further clarification or improvement:\n\n1.A table of contents should be added before the appendix to enhance readability and facilitate navigation for the readers.\n\n2.The comparison methods used in the paper are relatively outdated. More recent methods should be introduced for comparison to better demonstrate the effectiveness of the proposed approach.\n\n3.The performance of the proposed method is closely tied to the success of the unsupervised clustering process. When the clustering algorithm fails to adequately separate customers with different data distributions, the resulting model may underperform. This limitation warrants further exploration.", "was_revised": true, "final_justification": "Thanks to the author's response, I decided to maintain my score.", "rebuttal": "## Q1: Adding table of contents for appendix\n\nWe thank the reviewer for the suggestion. **We will add a table of contents to improve appendix navigation.** Here's the proposed structure:\n\n_Appendix Table of Contents:_\n| Section | Title |\n|--------|-------|\n| **A**   | Related Approaches to Distribution Shift |\n| **B**   | Overview of Baseline Algorithms |\n| **C**   | Comparative Analysis of Heterogeneity and Computational Costs |\n| **D**   | Privacy Implications of Distribution Descriptors |\n| **E**   | Implementation Details |\n| E.1     | Algorithm Pseudo-code |\n| E.2     | Code, Licenses and Hardware |\n| E.3     | Models and Hyper-parameter Settings |\n| E.4     | Descriptor Extractor |\n| E.5     | Identification of Distribution Shifts |\n| E.6     | Advantages of Latent Space Descriptors |\n| E.7     | Dynamic Clustering Initiation |\n| E.8     | Unsupervised Clustering Setup |\n| **F**   | Additional Experiment Results |\n| F.1     | Generating non-IID Datasets with ANDA and CheXpert |\n| F.2     | Scaling Data Heterogeneity Types and Levels |\n| F.3     | Scaling the Number of Clients |\n| F.4     | Robustness Under Mixed Distribution Shifts |\n| F.5     | Results on Real-World Dataset CheXpert |\n| F.6     | Ablation Study |\n\n\n## Q2: Choices of baselines\n\nWe appreciate this feedback and would like to clarify our baseline selection rationale. **Our comparison includes recent state-of-the-art methods from top-tier venues**: pFedMe (NeurIPS 2020), FedEM (NeurIPS 2021), ATP (NeurIPS 2023), FedDrift (AISTATS 2023), and FedRC (ICML 2024), **alongside established methods that remain competitive and widely cited in recent FL literature** (e.g., CFL, IFCA, APFL). We excluded some recent methods that, while novel, focus on specific subproblems or limited distribution shift types rather than the general problem of handling arbitrary distribution shifts that FLUX addresses.\n\nMore importantly, **we selected baselines based on capability relevance, not just recency**. As shown in Table 2, we systematically compare against methods across three key FL paradigms: Clustered FL (6 methods), Personalized FL (2 methods), and Test-Time Adaptive FL (1 method). This comprehensive coverage ensures we address all relevant approaches to handling distribution shifts in FL.\nOur experimental scope is extensive: 10 SOTA baselines across 4 benchmarks plus 2 real-world datasets (including OfficeHome, added during rebuttal), evaluating all 4 types of distribution shifts at 8 heterogeneity levels. **This experimental setup represents one of the most comprehensive CFL evaluations in recent literature.**\n\nWe welcome the reviewer to suggest specific recent methods they believe are missing, and we are happy to add new baselines and/or discuss their relevance to our problem setting of handling arbitrary distribution shifts with test-time adaptation capabilities.\n\n## Q3: How sensitive is FLUX to the quality of the unsupervised clustering?\n\nWe thank the reviewer for raising this important point. We fully agree that the performance of FLUX is influenced by the quality of the unsupervised clustering. **To systematically assess this sensitivity, we included FLUX-prior as an oracle variant that leverages access to the true number of clusters.** The relatively small gap between FLUX and FLUX-prior—e.g., 94.0% vs. 95.7% on MNIST (Table 1)—demonstrates that FLUX achieves near-optimal performance despite clustering imperfections.\n\nMoreover, FLUX remains highly robust even under strong heterogeneity, achieving 92.3% accuracy compared to 79.4% for the best baseline. This confirms FLUX’s resilience to moderate clustering imperfections, as its performance remains strong even when the clustering is not optimal.\n\n**To further validate robustness, we conducted an ablation study using four different unsupervised clustering algorithms on MNIST.** The results below show that our descriptors consistently support strong performance across all methods. Despite some variability—e.g., Unsupervised K-Means shows lower performance in the known-association setting (86.8%)—the test-time accuracies remain high across the board, ranging from 92.9% (HDBSCAN) to 94.2% (Agglomerative). **These findings confirm that FLUX performs robustly even when using alternative clustering strategies, and that its design is not tightly coupled to any specific clustering algorithm.**\n|Method|Known Association (%)|Test-Time (%)|\n|-|-|-|\n|U. K-Means|86.8±14.6|93.5±3.8| \n|HDBSCAN|90.5±7.7|92.9±2.6|\n|Agglomerative|93.9±3.2|94.2±0.8|\n|Ours|92.5±5.2|94.0±2.8|\n\nU. K-Means: Unsupervised K-Means clustering."}]

Author rebuttal:

Meta-review: This paper proposes a clustering-based FL framework FLUX which is designed to address data heterogeneity (distribution shift) across clients during both training and testing phases. This is achieved via leveraging a privacy-preserving descriptor extraction mechanism on clients to enable efficient unsupervised clustering and representation compression with low communication overhead. This design ensures robustness and scalability of the global model. It also enables adaptive model selection for unseen and unlabeled clients clients during the testing phase. Experiments on four benchmark datasets and one real-world dataset demonstrate that FLUX outperforms 10 state-of-the-art baseline methods under diverse distribution shift while maintaining computational and communication overhead comparable to FedAvg.

Following an extensive discussion with the authors, the reviewing panel recognizing the following key strengths of the paper: (1) novel label-free model assignment mechanism addressing key CFL limitations; (2) comprehensive evaluation across datasets and heterogeneity levels; (3) improved accuracy with FedAvg-comparable overhead; (4) well-structured presentation with strong empirical performance. 

There were also numerous questions raised by the reviewers regarding the motivation, theoretical foundation, additional experiment validation, and robustness to hybrid clustering (FLUX-soft), dynamic scenarios with client joining/leaving, and partial participation of clients. The authors have addressed all these questions sufficiently with an impressive and effective rebuttal. This results in an unanimous consensus on accepting this paper. This is a good paper that should be accepted.
Decision: Accept (poster)