Federated Learning with Profile Mapping under Distribution Shifts and Drifts
Authors: Mohan Li
Dario Fenoglio
Martin Gjoreski
Marc Langheinrich
Venue: ICLR 2026 Poster (2026)
Keywords: federated learning
privacy
distribution drifts
distribution shifts
data heterogeneity
efficiency
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: FEROMA is a distribution-aware Federated Learning framework that handles both distribution shifts and drifts without prior knowledge of data heterogeneity, achieving state-of-the-art performance with minimal overhead.

Abstract: Federated Learning (FL) enables decentralized model training across clients without sharing raw data, but its performance degrades under real-world data heterogeneity. Existing methods often fail to address distribution shift across clients and distribution drift over time, or they rely on unrealistic assumptions such as known number of client clusters and data heterogeneity types, which limits their generalizability.   We introduce **Feroma**, a novel FL framework that explicitly handles both distribution shift and drift without relying on client or cluster identity. **Feroma** builds on client distribution profiles—compact, privacy-preserving representations of local data—that guide model aggregation and test-time model assignment through adaptive similarity-based weighting. This design allows **Feroma** to dynamically select aggregation strategies during training, ranging from clustered to personalized, and deploy suitable models to unseen, and unlabeled test clients without retraining, online adaptation, or prior knowledge on clients' data. Extensive experiments show that compared to 10 state-of-the-art methods, **Feroma** improves performance and stability under dynamic data heterogeneity conditions—an average accuracy gain of up to 12 percentage points over the best baselines across 6 benchmarks—while maintaining computational and communication overhead comparable to FedAvg. These results highlight that distribution-profile-based aggregation offers a practical path toward robust FL under both data distribution shifts and drifts.

Published as a conference paper at ICLR 2026
FEDERATED LEARNING WITH PROFILE MAPPING UNDER DISTRIBUTION SHIFTS AND DRIFTS
Mohan Li", Dario Fenoglio", Martin Gjoreski & Mare Langheinrich Università della Svizzera italiana, Switzerland
ABSTRACT
Federated Learning (FL. enables decentralized model training across clients with- out sharing raw data, but its performance degrades under real-world data heterogene- ity. Existing methods often fail to address distribution shift across clients and distri- bution drift over time, or they rely on unrealistic assumptions such as known number of client clusters and data heterogeneity types, which limits their generalizability. We introduce FEROMA, novel FL framework that explicitly handles both distribu- tion shift and drift without relying on client or cluster identity. FEROMA builds on client distribution profiles- compact, privacy-preserving representations of local data-tha guide model aggregation and test-time model assignment through adap- tive similarity-based weighting. This design allows FEROMA to dynamically select aggregation strategies during training, ranging from clustered to personalized, and deploy suitable models to unseen, and unlabeled test clients without retraining, online adaptation, or prior knowledge on clients' data. Extensive experiments show that compared to 10 state-of-the-arr methods, FEROMA improves performance and stability under dynamic data heterogeneity conditions-an average accuracy gain of up to 12 percentage points over the best baselines across benchmarks- -hile maintaining computational and communication overhead comparable to FedAvg. These results highlight that distribution- cetnt--rroo-ebaaieee based aggregation offers practi- cal path toward robust FL under both data distribution shifts and drifts. Code is available ht https: /github. com/ da wwwarrrofooogoooooo / FEROMA.
INTRODUCTION
Federated Learning (FL. (McMahan et al., 2017) has become a promising paradigm for training models collaboratively across distributed clients without sharing their private data. However, one of the central challenges in FL is the presence of heterogeneous client data, which can significantly degrade model performance if not properly handled (Kairouz et al., 2021). In real-world deployments, clients rarely hold independent and identically distributed (IID) data (Zhao al., 2018; Zhu et al., 2021). They often exhibit two forms of heterogeneity: distribution shift, where different clients possess distinct data distributions (Sattler et al., 2021; Deng et al., 2020b; Guo et al., 2024), and distribution drift, where single client's data distribution also evolves over time (Jothimurugesan et al., 2023; Lu et al., 2019b; Gama et al., 2014). These dynamics challenge the notion of a single global model that performs uniformly well across all clients throughout training and deployment.
Vanilla FedAvg (McMahan et al., 2017) struggles to converge efficiently due to its uniform aggregation of all client updates, regardless of their data distributions. Under distribution shift (Figure 1 Left), clients whose data distributions are underrepresented in the global model receive limited benefit, resulting in persistently low accuracy and slow convergence. This issue is exacerbated under distribution drift (Figure 1 Right), where local data distributions evolve over time (In Figure 1, solid lines denote average accuracy across clients, while transparent lines correspond to individual client trajectories). Such drift may occur during global model-or during test time, degrading performance as clients encounter data that no longer aligns with the distribution observed during training.
instability and divergence in the
Existing methods for handling heterogeneous data in FL typically fall into three categories: Clustered FL (CFL) (Sattler et al., 2021; Guo et al., 2024; Jothimurugesan et al., 2023; Ghosh et al., 2020;
"Equal contribution.

---

Published as a conference paper at ICLR 2026
Marfoq et al., 2021; Long et al., 2023), Personalized FL (PFL) (Deng al., 2020b; T. Dinh et al., 2020; Tan et al., 2023a; Kulkarni et al., 2020), and Test-time Adaptive FL -TTA-FL) (Bao et al., 2023; Deng et al., 2020a; Wang et al., 2019). CFL methods group clients with similar data distributions based on model parameters or training metrics, which can effectively address distribution shift. However, they often lack adaptability under training and/or test-time drifts, require prior knowledge the number of clusters or assumptions about data distribution(Guu et al., 2024; Ghosh et al., 2020), and incur significant training overhead due to the use of computationally intensive clustering techniques, as well as the transmission, evaluation, or training of multiple models per client (Sattler et al., 2021; Jothimurugesan et al., 2023). PFL methods optimize personalized model for each client using its local data distribution, thereby mitigating the negative effects of inter- and intra-client distribution dissimilarity. While this yields strong performance parallyaati when ample training data available per the resulting models often become highly client-specific, sacrificing robustness in favour of personalization and offering limited generalization to unseen distributions or new clients. TTA-FL approaches are designed to handle test-time drifts, but usually rely on online adaptation or additional client interaction, which limits their practicality and efficiency in deployment. In addition, they often overlook training-time tiimingg-tmeeeee drift and shift, which can lead to unstable or slowed convergence and and degraded model performance across clients. Despite their strengths, these existing approaches are often tailored specifie non-IID types (Sattler et al., 2021: Deng et al., 2020b; Guo al., 2024; Jothimurugesan et al., 2023; Ghosh et al., 2020; Marfoq et al., 2021; Long et al., 2023; T. Dinh et al., 2020) and may struggle to balance generalization, adaptability, and efficiency- factors that are increasingly important for practical FL deployments under real-world heterogeneous conditions.
factors
To bridge this gap. we introduce Federated Learning with Distribution Profile Mapping (FEROMA), an FL framework that moves the focus from client or cluster identity to the un- derlying data distribution profile. To the best of our knowledge, FEROMA is the first general- purpose FL framework explicitly designed to ad- dress both distribution shift and distribution drift, during training as well as test time. FEROMA extracts a lightweight, differentially private sta- tistical profile from each client's i local data and maps it previously observed profiles from last training round. This mapping guides model ag- gregation through similarity-baas based weighting.
Distribution Shift
Distriburion Drift
1.0
FolAvg
0.8
Feroma
Accurncy
0.6
0.4
FedAvg
Peroma
0.0
Number Rounds
Namber Rounds
Figure 1: Comparison between FedAvg and FER- OMA under (Left) distribution shift across clients, and (Right) under distribution drift every rounds.
Based on these profiles, FEROMA A automatically selects the most suitable aggregation strategy for each round through adaptive similarity thresholds. This design enables FEROMA to remain both flexible and scalable under dynamic, real-world FL settings, without relying any prior knowledge. Moreover, FEROMA assigns trained models to unseen clients based on profile similarity, enabling robust model selection during test-time. We summarize the key contributions of FEROMA as follows:
A unified and adaptive aggregation framework. FEROMA dynamically selects the best aggre- gation craateyy-raginng from clustered to personalized or based on client distribution profiles, and naturally extends to test-time adaptation without retraining.
Effective under both distribution shift and drift. FEROMA handles static and dynamic data heterogeneity by leveraging round-wise distribution mapping, and scales efficiently to a large number of clients. We evaluate FEROMA on four standard and two real-world datasets, showing consistent gains over 10 SOTA baselines across a wide range of scenarios; including four types of distribution shift (with low, medium, and high severity) and varying drift frequencies.
Lightweight and efficient design. FEROMA introduces minimal communication and computation overhead, with both server- and client-side costs comparable to FedAvg. enabling practical deploy- ment in resource-conen constreeconstrainedd environments. We provide both theoretical bounds and empirical measurements to validate its practicality in resource-constrained federated settings.
BACKGROUND
FL under IID assumption. FL systems (McMahan et al., 2017) consist of a collection of K e N distributed clients, denoted IC (1,2,...,
.., K), coordinated by central server. These clients
2

---

Published as a conference paper at ICLR 2026
Training rounds
Testing round
R-1
2
Training clients
Cold-start
training clients
Test-only clients
Distribution shifts
Distribution drifts
Figure 2: Distribution shifts and drifts in FL. Colors indicate distinct local data distributions. Changes across clients reflect distribution shifts; changes over rounds reflect distribution drifts.
collaboratively train a shared machine learning model while keeping their local data private. Under IID assumptions, FL typically assumes that each client E K. holds an IID dataset sampled from common joint distribution P(X,Ya Y). Throughout, we use P to denote joint distributions over (X, Y).
Specifically, client draws a finite dataset ( {(x) Vj
(k)
(k) )) of size s(k) N; once sampled,
(*)) is treated as fixed throughout training. We write (A) E R (1)x and (A)) {0, 10 for the matrices collecting the inputs and labels in (k), where is the number of features per sample and u the number of output classes. During each communication round f, every client & performs local training by optimizing its model parameters 0(²) to maximize the likelihood over its private dataset (*), as shown in Equation 1. Once local updates are computed, clients transmit their parameters to the central server. The server then aggregates the received
via simple weighted averaging (McMahan al., 2017)-t0 produce new global model 0.+1 as defined in Equation 2. This updated global model broadcast back to the clients to initialize the next training round.
(Local training)
g(h)
arg max L(O (1),,, (x))
(1)
#
K
(k)
K
pk 0(*)
(Aggregation)
0++1
where PR
s6) Σ K
Pk
1.
(2)
FL under distribution shifts. In practical FL scenarios, clients commonly face inter-client distri- bution shift, where data distributions vary across ciients-referred to here as distribution shift. As illustrated in Figure 2 (vertical variation within each round), for any two clients k1 k2 E K., it may hold that p(k) + p(ka), violating the standard IID assumption p(k) P for all k. Such shifts may result from variations user behavior, environment, or adversarial activity, and are typically grouped into four types (Kairouz al., 2021):
Feature distribution skew (covariate shift): Marginal distributions P(X) vary across clients.
Label distribution skew (prior probability shift): Marginal distributions P(Y) vary across clients.
Concept shift (same X, different Y): Conditional distributions P(Y X) vary across clients.
Concept shift (same Y, different X): Conditional distributions P(X Y) vary across clients.
While distribution shift is well-studied in centralized learning (Lu et al., 2019b; Koh et al., 2021; Li al., 2022c; Tahmasbi al., 2021b), it remains underexplored in FL-where most works target specific shift types and demand extra communication/compaaatt al., 2020b; Guo et al., 2024: Ghosh et al.. 2020; Marfoq et al., 2021: Long al., 2023: T. Dinh et al.. 2020; Tan et al., 2023a; Kulkarni et al., 2020), due to the server's 's limited visibility into decentralized client data. This challenge is further compounded by resource constraints on client devices, which limit both the training capacity and the complexity of deployable models. As result, global models often fail to generalize across non-identically distributed client populations.
costs (Sattler et al., 2021; Deng
FL under distribution drifts. In addition distribution shift, FL systems may also encounter intra- client distribution drift, which we refer to as distribution drift. As illustrated Figure (horizontal variation across rounds), this occurs when the local data distribution of single client changes
3

---

Published as a conference paper at ICLR 2026
time. Formally, for any client k that participates at both rounds & and tz (i.e., k Kl, ((g)), the local distribution may drift over time: p!A) + p(()) Drift may manifest during training-dus to changes in user behavior, sensor conditions, data collection environments- or during testing. In the latter case, drift arises either because the same client exhibits evolving behavior at inference time or due to the presence of entirely new clients (e.g., test-only clients in Figure 2) whose data distributions were not observed during training and for which label information is unavailable. Such temporal drift significantly increases training and testing complexity and necessitates adaptive strategies that can respond to evolving data. While distribution drift has been extensively studied in centralized settings (Lu et al., 2019a; Gao et al., 2022: Li et al., 2022b; Tahmasbi et al., 2021a), only a few studies have investigated training-ting -amminggttmeeeee drift (Jothimurugesan et al., 2023; Chen et al., 2024b) or test-time drift (Bao et al., 2023; Tan et al., 2023c) in FL. However, to the best of our knowledge, the joint presence of training and test-time drift has never been explored in existing FL pherature-despicc its critical impact on model performance and accuracy in real-world deployments.
FEROMA
To tackle the challenges posed by dynamic and heterogeneous federated environments, we propose FEROMA, lightweight framework that adapts to both distribution shift and drift. This section begins by formalizing the problem setting (section 3), then outlines the two core components of the FEROMA pipeline, illustrated in Figure 3: distribution profile extraction (subsection 3.1) and distribution profile mapping (subsection 3.2). In addition, section 3.2.1 details how FEROMA A dynamically selects the optimal aggregation strategy for each client based on the inferred distribution structure. The full implementation and algorithmic details are provided in Appendix B.
Problem definition. We consider a dynamic FL system with Ke clients at round I, denoted as K, = {1...., [I....,KK),, Kr}, where each client holds local data D (z. yi (()) and trains local model o(()) Each (²(²) is drawn from an underlying local distribution p!(e over (X, Y) and treated as fixed at round once collected. The number of clients may vary across rounds due to client arrivals (cold-start) or departures, i.e., Ki-1 * Kg- In practical FL settings, data distributions are both non-identical across clients and non-stationary over time. Formally, for any 4, and any ki E K.
22 E Kag, the local distributions may differ: p((k) + p((k2), This unified formulation captures both inter-client shift (when b # 12) and intra-client drift (when b = k2). Our goal is to design a lightweight FL framework that explicitly addresses both distribution shift and drift during training and testing, without relying on client identities or prior knowledge of underlying distributions.
3.1 DISTRIBUTION PROFILE EXTRACTION
When a client's local distribution shifts between consecutive rounds-e.3 from P.()) to p(A) adapting a model trained on its own earlier distribution may require significant local computation and communication. A more efficient alternative is reassign a model (or an aggregation of models) previously trained on distributions that closely resemble the client's current distribution. To enable such model selection under distribution drift and to guide aggregation across heterogeneous clients under distribution shift, we require a distribution profile, i.c., a low-dimensional, stable summary that quantifies distribution similarity without exposing raw data or labels. Our use of compact distribution summaries builds on the descriptor- perspective introduced in (Fenoglio et al., 2025), while targeting drift-aware aware temporal matching across rounds.
Definition 3.1 (Distribution Profile Extractor). Let z E N denote the feature dimension, u E N the label dimension, and let d be the desired profile dimension. A Distribution-Profle Extractor (DPE) is a stochastic mapping dh R (2+a) - R2, with vi
(k) < st parametrized by th, that maps the
(R)
dataset () to a distribution profile d(A)) :- ow(( (D((t)) Intuitively, the DPE compresses client's dataset into a compact distributional signature that captures how the data is distributed, rather than the data itself, enabling reliable similarity estimation without exposing fine-grained information. For any two client-round puirs (ki, (1) and (k2, l2), the extractor musl sutisfy the requirements below.
(R1) Distribution fidelity. Profile distances should approximate a reference distance A (e.g., Jensen- Shannon or Wasserstein) between the corresponding underlying distributions. Concretely,
4

---

Published as a conference paper at ICLR 2026
letting the expectation be taken over the internal randomness of Φs.
[ d(ft) ll2 - A(P(.))) ,P(())) s.
In other words, the extractor Φy should map similar distributions to nearby profiles and dissimilar distributions distant profiles.
(R2) Label agnosticism. The extractor de must support profile generation without labels:
de = the (at (k)
0) E IR, with S d,
where d(()) is subvector of the full profile d!a) R. This enables profile extraction and similarity matching at test time. Here, d(() captures marginal distributional characteristics (k)
based solely the input features xt
(R3) Controlled stochasticity. The extractor oy is a stochastic mapping: for fixed input dataset, the profile deth) (D())) is random vector. Let the expectation taken over the internal randomness of de. we have Eld!") DD = (D( with bounded conditional covariance Cov(d, d(a)) 1 3 I²I This controlled stochasticity prevents exact fingerprinting of client distributions across rounds while maintaining reliable inter-profile distances in expectation.
(R4) Differential-privacy guarantee. The mapping dh. satisfies (e, (2,6))dinn -idfffrrrtt privacy at the
sample level: for any two datasets (( and D(()) that differ in exactly one example and for every measurable set S C Rd.
Pr(o ES] S ef Pr(ou(D(()) SS] +
For example, this guarantee can be realized with a Gaussian (or Laplace) mechanism: d! Ty (D(x)) + -(0,, I²I,). where a is calibrated to the fz-sensitivity of de. The resulting many-to-one mapping both limits information leakage and obfuscates client identity.
(R5) Compactness. Profile extraction introduces minimal overhead compared to vanilla FL: its computational cost is negligible relative to local training epoch, and the profile dimension d satisfies d < |0] (typically d/|00 10-3), ensuring that the additional communication cost remains marginal compared to the transmitted model update.
We implemented our DPE using a four-step statistical moment extraction from latent space with differential privacy. We provide details of the implemented DPE in our FEROMA in Appendix B.4, which satisfies all five requirements: (R1) with mapping Φv provably bi-Lipschitz equivalent to the 2- Wassersteei metric, showing & 1.1 on MNIST (and < 0.54 under Jensen -hnsen-Shannon); (R2) by
consistently providing a label-free subvector ((()) that approximates the marginal data distribution;
(R3) with bounded covariance p² = Myoutt + 212 maxx < 2.2x 10 depending on a Monte-Carlo subsampling (M,7) plus Laplace noise (dmax); (R4) by ensuring (e. O)-DP for each profile vector d( with added variance < 2.2 x 10 5, and (R5) by introducing negligible computation and a communication cost of d/|0] < 3.5 x 10-3. Full implementation details and theoretical justifications are Appendix B.4, with privacy calibration in Appendix F.
,2
3.2 DISTRIBUTION PROFILE MAPPING
The core idea of FEROMA to decouple model identity from specific clients clusters, and instead associate each model with a data distribution characterized by its distribution profile d(*) Once profiles are extracted, employ model sharing by matching current and past profiles to derive weighted aggregations across clients; during testing, we extract a label-free profile for each unseen client and assign the closest model from the final round for direct inference.
complementary mapping strategies: during training, enable
3.2.1 TRAINING DISTRIBUTION MAPPING
After extracting all distribution profiles for the current round i.c., {d(t last round profiles {a{()) {d--1)ke to define the weights for model aggregation and assignment. The
(d(5)) == we map them to the
5

---

Published as a conference paper at ICLR 2026
Training round
Training round
Training round
Round
Model
Weighted Aggregation (M)
Σ, 1l
Local Training 9,"
Lapoord prfics
Rg
19:
0/-00
DPM
0.10
0.36
0.10
0.02
Pldrrbatton Profile Mapping
DPM
0.05
0.24
0.07
0.5
0.11
0.19
DPF
0:41
8:17
0.10
0.04
5.08
DPB
0.07
2.58
Cumme-mund postler
(.
09
Figure 3: FEROMA pipeline. In each round I, clients extract distribution profiles (DPE), map them to previous-round profiles (DPM), and compute weighted aggregation (WA) for local training (LTT).
mapping can be done with a normalized distance function:
(D(d (5) ((()))))))))
exp
(k.j) Wi
Σ exp (DD(d))) .d41)) J'E
where :):,)) is chosen distance function (e.g., Euclidean distance), w((a]) between current client & and previous-round client j. and A the set of clients active round 1.
(k.j) the association weight
Although weakly similar profiles receive small weights, their aggregation still introduce noise. To regulate when collaboration is beneficial, threshold can be applied discard any w (k.j) below т, ensuring that model updates are shared only among sufficiently similar profiles and preventing both over-collaboration and unnecessary fragmentation, in manner similar clustered FL approaches:
(kJ) Wx
w
(k.j)
if W. (k.J) 2T
(4)
otherwise
After thresholding, the weights w, (k.s) are renormalized to ensure they sum across j E A.-1- This weight can be combined with weighting based on data size in FedAvg. More details on the distance function -)..) -) and ablation studies on the utility are in Appendices 1,9 and I.10.
Automatic aggregation strategy selection. At each training round f, we compute the association weights {w] (xr( JuEKu jEA,, via equation or the thresholded wh (k(k) version via equation depending on the desired level of selectivity, and then aggregate as
0!
Σ JEAc-1
(kj) W
0
(5)
or wt (k.)) After aggregation, each client proceeds with its local training step as in Equation 1. By inspecting the support of {w] (k:)) JjA.. for each client k at round t, FEROMA independently and dynamically recovers the most suitable FL aggregation strategy:
(k.j) where w (k.j) denotes either w.
Clustered FL. (a.") in Figure 3) If multiple weights w (k.) > survive thresholding, the aggregation
for client k aggregates those models trained on similar data distribution, akin to CFL.
Personalized FL. (d dftt,, d(5), ((()) in Figure 3) If exactly one thresholded weight is nonzero, client & simply inherits that single most similar model, yielding personalized specialization.
Global FL fallback. (d(2) Figure 3) no sufficiently similar profile identified, it falls back to
global aggregation (w) next adaptive round.
(kJ) = 1/|AA-1 if not thresholded), thus combining all models before the

---

Published as a conference paper at ICLR 2026
3.2.2 TESTING DISTRIBUTION MAPPING
At test time, we aim to assign each unseen client k the best-matching model learned in the final training round At based on its feature distribution, without any further optimization. First, we extract the label-free profile d'ch) de (areed 0), required by (R2). Then match d'st against the set of round- R profiles { IEEAR by selecting the nearest neighbor in profile space:
j arg min D(drost
JEAR
test
This one-shot assignment requires gradient steps, leverages the DP pr-pteteteed distribution profiles, and naturally generalizes to unseen, unlabeled clients. As discussed in Appendix E, pure label- free matching cannot inherently capture concept shift with identical X but different Y. However, for addressing this problem, we show that a small, test-time labeled validation set can seamlessly refine the associations and substantially improve performance. In addition, by assigning the most appropriate pre-trained model to each test client, FEROMA enables the integration of unsupervised test-time adaptation methods by offering distribution-awv aware initialization point (see subsection 5.2).
EXPERIMENTS
This section presents our experimental setup and results from two primary scaling studies. We evaluate FEROMA under varying drift frequencies, non-lID types, severity levels, and numbers of clients. These experiments assess the scalability, robustness, and efficiency of FEROMA compared to 10 SOTA baselines across a wide range of real-world heterogeneity scenarios.
4.1 EXPERIMENT SETTINGS
Drifting datasets generation. We employ six publicly available datasets for our experiments: MNIST (LeCun & Cortes, 2005), Fashion-MNIST (FMNIST) (Xiao et al., 2017), CIFAR- 10, CIFAR- 100 (Krizhevsky), and two real-world datasets, CheXpert (Irvin et al., 2019) and Office- Home (Venkateswara al., 2017). To construct distribution shift and drift datasets under different non-IID conditions, we use ANDA a toolkit supporting operations such as class isolation and label swapping. For the real-world datasets, we preserve their intrinsic characteristics without modification. Detailed dataset information is provided in Appendix I.I.
Baseline algorithms. We evaluate our approach against baseline methods summarized in Table including: FedAvg (McMahan et al., 2017), FedRC (Guo et al., 2024), FedEM (Marfoq et al., 2021), FeSEM (Long al., 2023), CFL (Sattler et al., 2021), IFCA (Ghosh et al., 2020). FedDrift (Jothimurugesan et al., 2023), pFedMe (T. Dinh et al., 2020). APFL (Deng et al., 2020b), and ATP (Bao et al., 2023). The baseline methods are detailed further in Appendix A. The experimental environment, models, and hyperparameter configurations are described in Appendices B.2 and B.3.
4.2 RESULTS
Scaling the drifting frequency, non-lID types, and non-lID levels. We first evaluate the robustness of FEROMA by comparing it against baseline methods across three drift frequencies: each client's dataset drifts every four rounds, every two rounds, and at every round. Additionally, we simulate four types of distribution (iifts-P(X)) P(Y). P(Y|X), ), and P(X cach under three levels of non-lID severity: low, medium, and high. This setup enables us to thoroughly assess the adaptability of FEROMA in highly dynamic and heterogeneous FL environments, Detailed experimental setups and results are provided in Appendices I.2 and I.4, and summarized in Table 1. Ablation study on training rounds is summarized Appendices and K.
Table shows that FEROMA consistently outperforms all baselines across six benchmark datasets, with results computed as unweighted averages over drift frequencies, non-IID types, and severity levels, without case selection. Notably, FEROMA improves accuracy by up to 14.1, 14.3, and 10.3 percentage points (pp) over the best- performmng baseline CFL on MŇIST, FMNIST, and CIFAR-I0. respectively. On CIF,R-100, it achieves an accuracy of 39.9%, surpassing FedEM by 8.2 pp. On
'https://gi thub. com/ / Fredolii mc/ ANDA

---

Published as a conference paper at ICLR 2026
the real-world datasets CheXpert and Office-Home, FEROMA demonstrates consistent robustness, exceeding the strongest baselines while maintaining lower variance. Table further shows that FEROMA remains robust under varying scales of distribution shift and drift (see Appendix 1.2 for detailed results of additional benchmarks). These improvements are achieved under realistic FL conditions with no prior knowledge of distribution modes or test-time labels. While haselines are often specialized for either shift or drift, FEROMA adapts to both with significantly lower variance. The results underscore the effectiveness of bestribution-profle-bbiee as generalizable solution for dynamic non-IlI -nDIID FL conditions.
aggregation and highlight FEROMA
Scaling the number of clients. In real-world FL deployments, the num- ber of participating clients can be large, requiring FL. methods to remain scalable with minimal computational and communication wverhead-ever even under highly dynamic conditions. To evaluate scalability, we assess the per- formance of FEROMA and baseline methods with MNIST the num- ber of clients increases from 10 to 20, 50, and 100. Notably, FedDrift could not be evaluated with 50 or 100 clients due to excessive computational requirements (see Appendix B.2 for details). A summary of the results is presented in Figure 4, with detailed experimental setup and results provided in Appendix I.3.
Dataset
MNIST
FMNIST CIFAR-I0
CheXpert Office-Home
FodAvg
71.8+55
63.7+6.4
33.0+5.3
28.2+47
4,30 $9.1
FodRC
3099699
45.1
23.2+4,5
30.6+42
15.1
14.6+3.
FedEM
30.7+7.0
46.1
23.0+48
31.7436
53.3+2
15.5
ReSEM
69.625.7
59.055.6
31.1488
26.24377
61.0=199
33.820.9
CFL
76.61399
65.6148
33.924.7
28.9335
62.3222
348810
IFCA
44.1
369992
27.44.7
15.7+46
5288
33.5
PFedMe
53.2+74
43.6+68
24.24..1
158+22
1..2+1.1
3.66
APFL
70.0
$6.955.8
31.424.5
29.743.1
$4.4
39,4
FedDDrftt
$7.0
47.6
29.2
20.2
7..3
42.1
ATP
12.1 12.1
28.7=5.1
16.7
NIA
40.8
FEROMA 9..7 79.928
44.2+388
3..9
72.4206
4.44
Table 1: Mean accuracy and standard deviation across dif- ferent datasets, comparing FEROMA and baselines under varying drifting frequency, non-lID types and levels.
Non-IID Level
Low
Medium
High
Drifting
5/20
10/20
20/
5/20
10/
5/20
10/ 20
20 00//2 20
FedAve
8..1
4,7
2.6
71.5 1.5
5..
75.8 2.6
63.5 + 6.6
65.1
68.6 6.5
FedRC
4..7
7.3
448
23.8
26.6
2.1
22.9
FedEM
40.3
7,3
44.5 10.5
21.7 4.8
2.28
2.17
2.92
28.4 2.2
FoSEM
7.74
7..8
6.
8.
60.2662
6.66688
64.4
CFL
29,3
7.0
8.0
76.6
78.
7.9
69.5
3.7
2.9
IFCA
51.0+99.
52.9 10.9
13.0
12.1122.1
8.7
4..5
4..6
8.4
8,84
pPedMe
7..9
7.0
59.5
93.
7.2
5.06
57.
4.2
4.
APFL.
69.0
4.74
7.21
70.0
718
7..9
64.5
5.3
6.0
FedDrift
58.1 9.0
60.8
6.06
6.0
6.87
5.5
ATP
70.8 12.8
78.5
10.7
83.9
65.8
14.2
7..6
8.7
$9.1
65.1
2
11.5
70.428.4
FEROMA
9..6
92 914
2.7
90.6
91
90.2
8.6
Table 2: Performance comparison across three non-llD levels and three drifting levels of all non-lID types on the MNIST dataset. 5 / 20, 10 / 20, 20 / 20: Drifting 5 / 10 / 20 times in overall 20 rounds.
1.0
2/4
Despite the increased data heterogene- ity and reduced data size introduced by scaling up the client number, Fig- ure 4 shows that FEROMA consistently achieves the highest accuracy across all cettings-exceeen
Pubkay
212
0.8
Time (secondss
Accuracy
0.6
210
90% with even 50 clients, and maintaining over 85% accu- racy with 100 clients. It outperforms the best baseline CFL by more than 10 pp at the largest scale. In addition to accuracy. FEROMA demonstrates strong compu- tational and communication efficiency, with training times comparable to Fe- dAvg. In contrast, most other baselines experience sharp increase in runtime as the number of clients grows. This ef- ficiency is attributed to the lightweight nature of distribution profiles which introduce minimal overhead throughout the training process. The results underscore the robustness and scalability of FEROMA in large-scale, dynamic FL scenarios.
28
0.4
0.2
Perteta
TocEC RTC
2
0.0
20
50
100
10
20
100
Number of clients
Number of clients
Figure 4: Performance comparison across varying numbers of clients. Left: Mean accuracy and standard deviation. Right: Training time per 20 rounds.
8

---

Published as a conference paper at ICLR 2026
Algorithu
Car.
D. shift
D. drift
adapt.
Low comn..
Low comp. (Server)
Low comp. (Client)
Scalability
FedAvg
N/A
SSO OOK
FedRC
CFL
FedEM
FcSEM
CFL
CFL
CFL
IFCA
CFL
PFodMc
PFL
APEL
PEL
FedDrift
CFL
TTA-FI.
FedProto
PFL
FedGPS
PEL
FOOGD
PFL
ShittEx
PEL
CoLA
TIA-FL
N/A
NIA
NIA
FEROMA
N/A
Table 3: Qualitative comparison among FEROMA and baselines. Cat.: FL category. D. shift/drift:
Designed to tackle distribution shift/drift. T. adapt.: Designed to adapt test time distribution. Low comm.: Low communication cost (comparable to FedAvg). Low comp. (Server/Client): Low computational
server/client side (comparable to FedAvg). Scalability: Scales efficiently to large client number. CFL: Clustered FL. PFL: Pensonalized FL. TTA-FL: Test time adaptive FL. v: Q: Property conditionally satisfied.
Property satisfied.
Key Observations. FEROMA demonstrates consistent robustness across datasets, non-lll severities, and drifting frequencies. By dynamically selecting aggregation behavior at each round, it effectively recovers clustered or personalized FL when client distributions are similar, and falls back to global aggregation when distributions diverge sharply. This adaptability allows FEROMA maintain strong performance in highly heterogeneous and unstable settings where fixed-strategy baselines degrade.
5 DISCUSSION
5.1 RELATED WORKS
FEROMA relates to three major lines of work in FL designed to address data heterogeneity: CFL, PFL, and TTA-FL. CFL methods (Sattler et al., 2021; Guo et al., 2024; Jothimurugesan et al., 2023; Ghosh al. 2020; Marfoq et al., 2021; Long et al., 2023) group clients with similar data distributions and aggregate models accordingly. While effective under distribution shift, they typically assume fixed number of clusters (Guo et al., 2024: Ghosh al., 2020), require computationally intensive cluster procedures, and involve transmitting or maintaining multiple models per client (Jothimurugesan et al., 2023), limiting scalability.
contrast, FEROMA avoids explicit clustering by leveraging continuous distribution profiles for soft, data-driven association. PFL methods (Deng et al., 2020b; T. Dinh et al., 2020; Tan al., 2023a; Kulkarni et al., 2020; Zhang et al., 2020a; Marfoq al., 2022; Tan al., 2022; Yang et al., 2025; Liao et al., 2024; Bhope et al., 2025) personalize models to each client's local distribution, improving performance when sufficient local data is available. However, they incur higher computation and storage costs the client side and lack mechanisms for model assignment in cold-start or test-time scenarios. FEROMA achieves similar personalization effect when required, by matching profiles without per-client optimization, and with significantly lower system overhead. TTA-FL methods (Bao et al., 2023; Deng et al., 2020a; Wang et al., 2019; Liang et al., 2025; Rajib al.: Chen et al., 2024a) are designed handle test-time drift via online adaptation, which requires additional client interaction or retraining. FEROMA, by contrast, supports test-time adaptation by matching profiles to observed ones, requiring no further updates or communication. We summarize the qualitative comparison in Table 3 and provide more details in Appendix A.
5.2 LIMITATIONS AND FUTURE WORKS
Extractor dependence. While prior works represent client distributions using model parameters, gradients, or training metrics-whiih exhibit intrinsic limitations (see Appendix A)-the effectiveness of FEROMA similarly depends on the quality of its DPE, which must generate reliable representations of client data distributions. In implementation, the DPE relies to embed sampled data into a latent space. This approach may be limited in two scenarios: (1) if the model is undertrained e.g., due difficult task or limited data the resulting latent space may
few-round pretrained model

---

Published as a conference paper at ICLR 2026
adequately reflect the underlying distribution; (2) if the model is overly simplistic overly complex, the extracted representations may be uninformative or unstable. In both cases, suboptimal profiles may impair the accuracy of distribution mapping and reduce the overall robustness of FEROMA.
Unseen Distributions. FEROMA associates models with training-time data distributions, but it does not explicitly address two challenging scenarios: (1) distributions that were seen during training but are absent in the final round, and thus not retained: and (2) entirely unseen distributions at test time. (See Appendix for further discussion) However, FEROMA demonstrates strong generalization, as it assigns the most relevant model based profile similarity-evve for distributions not directly observed during training. Moreover, the models associated with final-round profiles offer strong initialization for downstream personalization unsupervised adaptation. Unlike methods that begin personalization from a generic global model, FEROMA provides well-trained, distribution-aww -warre starting point. Future work could periodically checkpoint model in sparsely populated regions of the descriptor space, ensuring that rare or transient distributions are retained for test deployment.
CONCLUSIONS
In this work, we proposed FEROMA, an FL framework that explicitly addresses both distribution shift and drift across all four major types of data heterogeneity. By leveraging lightweight, differentially private distribution profiles to represent client data, FEROMA enables adaptive model aggregation based distributional similarity without relying any prior knowledge. This profile-based design supports both training and test-time adaptation, allowing FEROMA to generalize across dynamic client populations and unseen distributions, without requiring retraining or personalization from scratch. Through extensive experiments, we demonstrated that FEROMA consistently improves robustness and performance across wide range of non-III scenarios, with minimal overhead. Unlike prior methods that specialize in clustered, personalized, or adaptive FL, FEROMA unifies these strategies under a single tramework-scalable adaptable, and suitable for real-world heterogeneous deployments. This work lays the foundation for distribution-drive FL, and opens new directions for profile-base personalization, distribution tracking, and generalization to unseen client data in future systems.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary. Feder- ated Learning with Personalization Layers, December 2019.
Wenxuan Bao, Tianxin Wei, Haohan Wang, and Jingrui He. Adaptive Test-Time Personalization for Federated Learning. Advances in Neural Information Processing Systems, 36:77882-77914, December 2023.
Daniel J. Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Javier Fernandez-Marques, Yan Gao, Lorenzo Sani, Kwing Hei Li, Titouan Parcollet, Pedro Porto Buarque de Gusmão, and Nicholas D. Lane. Flower: A Friendly Federated Learning Research Framework, March 2022.
Rahul Atul Bhope, K. R. Jayaram, Praveen Venkateswaran, and Nalini Venkatasubrammaiian Shift happens: Mixture of experts based continual adaptation in federated learning, 2025, https:
//arxiv. org. abs 220. 18789.
Luxin Cai, Naiyue Chen, Yuanzhouhan Cao, Jiahuan He, and Yidong Li. FedCE: Personalized Federated Learning Method based on Clustering Ensembles. In Proceedings of the s1 ACM International Conference on Multimedia, MM '23, pp. 1625-1633, New York, NY, USA, October 2023. Association for Computing Machinery. ISBN 9798400701085. doi: 10.1145/3581783. 3612217.
Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive Test-Time Adaptation. In 2022 JEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 295-305, New Orleans, LA, USA, June 2022. IEEE. ISBN 978-1-66546-94622 doi: 10.1 10.15099 2022.00039.
Fei Chen, Mi Luo, Zhenhua Dong, Zhenguo Li, and Xiugiang He. Federated Meta-Learning with Fast Convergence and Efficient Communication, December 2019.
Guohao Chen, Shuaicheng Niu, Deyu Chen, Shuhai Zhang, Changsheng Li, Yuanqing Li, and Mingkui Tan. Cross-device collaborative test-time adaptation. In A. Globerson, L. Mackey, D. Bel- grave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Pro- cessing Systems, volume 37, pp. 122917-122911. Curran Associates, Inc., 2024a. doi: 10.52202/ httpp//eeecpppeeeeeeeeepepeep 2024 0000///2166000111111366688668000
079017-3906. URL https:
Junbao Chen, Jingfeng Xue. Yong Wang. Zhenyan Liu, and Lu Huang. Classifier Clustering and Feature Aligmment for Federated Learning under Distributed Concept Drift. Advances in Neural Information Processing Systems, 37:81360-88 81388, December 2024b.
Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. Exploiting Shared Represen- tations for Personalized Federated Learning. In Proceedings of the 38th International Conference on Machine Learning, pp. 2089-2099. PMLR, July 2021.
Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi. Adaptive Personalized Federated Learning, November 2020a. URL http:://aaxii.og/ab////////ss 134.610...31..1.. arXiv:2003.1 13461 [cs, stat].
Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi. Adaptive Personalized Federated Learning, November 2020b. URL arXiv:2003. [cs, stat].
Enmao Diao, Jie Ding, and Vahid Tarokh. HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients. In International Conference on Learning Repre- sentations, October 2020.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating Noise to Sensitivity in Private Data Analysis. In Shai Halevi and Tal Rabin (eds.). Theory of Cryptography, pp. 265-284, Berlin, Heidelberg, 2006. Springer. ISBN 978-3-540-32732-5. doi: 10.1007/1 168 1681878114 14.
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized Federated Learning: A Meta- Learning Approach, October 2020.
11

---

Published as a conference paper at ICLR 2026
Dario Fenoglio, Gabriele Dominici, Pietro Barbiero, Alberto Tonda, Martin Gjoreski, and Mare Langheinrich. Federated Behavioural Planes: Explaining the Evolution of Client Behaviour in Federated Learning. In The Thirty-eighth Anmual Conference Neural Information Processing Systems, November 2024.
Dario Fenoglio, Mohan Li, Pietro Barbiero, Nicholas D. Lane, Mare Langheinrich, and Martin Gjoreski. FLUX: Efficient Descriptor-Driven Clustered Federated Learning under Arbitrary Distribution Shifts. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, October 2025.
João Gama, Indre Žliobaité, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. survey concept drift adaptation. ACM Computing Surveys, 46(4):1-37, April 2014. ISSN 0360- 0300, 1557-7341. doi: 10.1145/2523813, URL https: /dl acm acm.aam..... www.aaa......................... awg/doi 114.5 2523813.
Hongchang Gao, An Xu, and Heng Huang. On the Convergence of cottmmunication-ffffiiii Local SGD for Federated Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 35(9): 7510-7518, May 2021. ISSN 2374-3468. doi: 10. 10.1609/aaai v35i9. 16920.
Yang Gao, Swarup Chandra, Yifan Li, Latifur Khan, and Thuraisingham Bhavani. SACCOS: A Semi-Supervised Framework for Emerging Class Detection and Concept Drift Adaption Over Data Streams. IEEE Transactions on Knowledge and Data Engineering, 34(3):141611266 March 2022.
ISSN 1558-2191. doi: 20.119//KDE.2020.29999
Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran. An Efficient Framework for Clustered Federated Learning. Advances in Neural Information Processing Systems, volume 33, pp. 19586-19597. Curran Associates, Inc., 2020.
Biyao Gong, Tianzhang Xing, Zhidan Liu, Wei Xi, and Xiaojiang Chen. Adaptive Client Clustering for Efficient Federated Learning Over Non-IID and Imbalanced Data. IEEE Transactions on Big Data, 1066):0511-106; December 2024. ISSN 2332-7790. doi: 10. 0011000BBDXTA.202231179999
Yongxin Guo, Xiaoying Tang, and Tao Lin. FedRC: Tackling Diverse Distribution Shifts Challenge in Federated Learning by Robust Clustering, June 2024.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattenn Recognition (CVPR), pp. 770-778, Las Vegas, NV, USA, June 2016. IEEE. ISBN 978-1-4673-8851-11 doi: 10.1109/ 2016.90. URL http::ttt//..... /tttt::://eeeeeplooeeeee
feee .rre www.rr//oooeee////oooooo
Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. Deep Models Under the GAN: Infor- mation Leakage from Collaborative Deep Learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS '17, pp. 603-618, New York, NY, USA, October 2017. Association for Computing Machinery. ISBN 978 978-1-4503-4946-88 doi: 10.1145/31339566113401
J. D. Hunter. Matplotii: A 2D graphics environment. Computing in Science & Engineering, 9(3): 90-95, 2007. doi: 10.1 10110/9/...22000.55
Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ileus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, Jayne Seekins, David A. Mong, Safwan S. Halabi, Jesse K. Sandberg. Ricky Jones, David B. Larson. Curtis P. Langlotz, Bhavik N. Patel, Matthew P. Lungren, and Andrew Y. Ng. CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. Proceedings of the AAAI Conference on Artificial Intelligence,
597, July 2019. ISSN 2374-3468. doi: 10.1609/aaii.v333 v33101 1.3011590 URL https: /ojs aaai. http://www..pp//AAaaaaaa.eeeee// 01.
Number:
Liangze Jiang and Tao Lin. Test-Time Robust Personalization for Federated Learning. In The Eleventh International Conference on Learning Representations, September 2022.
Yihan Jiang. Jakub Konečný, Keith Rush, and Sreeram Kannan. Improving Federated Learning Personalization via Model Agnostic Meta Learning January 2023.
12

---

Published as a conference paper at ICLR 2026
Ellango Jothimurugesan, Kevin Hsieh, Jianyu Wang, Gauri Joshi, and Phillip B. Gibbons. Federated Learning under Distributed Concept Drift. In Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, pp. 1834-5853. PMLR, April 2023.
Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. D'Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He. Lie He. Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konecný, Alcksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgüir, Rasmus Pagh, Hang Qi. Daniel Ramage, Ramesh Raskar, Mariana Raykova, Dawn Song. Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao. Advances and Open Problems Federated Learning. Foundations and Trends@ in Machine Learning, 14 (1-2):1-210) June 2021. ISSN 1935-8237, 1935-8245. doi: 10.1561/2200000000
Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsub- ramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran Haque, Sara M. Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. WILDS: A Benchmark of in-the-Wild Distribution Shifts. In Proceedings of the 38th International Conference on Machine Learning, pp- 5637-5664. PMLR, July 2021.
Anastasia Koloskova, Nicolas Loizou, Sadra Boreini, Martin Jaggi, and Sebastian Stich. A Unified Theory of Decentralized SGD with Changing Topology and Local Updates. In Proceedings of the 37th International Conference on Machine Learning, pp. 5381-5393, PMLR, November 2020.
Alex Krizhevsky. Learning Multiple Layers of Features from Tiny Images.
Viraj Kulkarni, Milind Kulkarni, and Aniruddha Pant. Survey of Personalization Techniques for Federated Learning. In 2020 Fourth World Conference Smart Trends in Systems, Security and Sustainability (WorldS4), pp. 794-797, July 2020. doi: 10.110000ord55550033220000000000
URL https:://///mm /eteeeeeeeeeeeeeeeeeeeeeeeeeeeee ieee org/ http://bbbttat///ocuuueuttee /21/ 0355.
Y. Lecun, L. Bottou, Y. Bengio, and Haffner. Gradient-based learning applied document recogni- tion. Proceedings of IEEE, 86(11):2278-2) 23244. November 1998. ISSN 1558-2256. doi: 10.1 5.726791. URL https: wwweeeeeeperreeeeeee ieee httpe/////.aaaaaccc/ccccttt///// Conference Name: Proceedings of the IEEE.
Yann LeCun and Corinna Cortes. The mnist database of handwritten digits. 2005.
Jiacheng Li, Ninghui Li, and Bruno Ribeiro. Effective passive membership inference attacks in federated learning against overparameterized models. The Eleventh International Conference on Learning Representations, September 2022a.
Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. Ditto: Fair and Robust Federated Learning Through Personalization. In Proceedings of the 38th International Conference on Machine Learning, pp. 6357-6368. PMLR, July 2021a.
Wendi Li, Xiao Yang. Weiqing Liu, Yingce Xia, and Jiang Bian. DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptation. Proceedings of the AAAI Conference on Artificial Intelligence, 36(4):4092-4100, June 2022b. ISSN 2374-3468. doi: 10.1609/aaai.v36i6 20327.
Wendi Li, Xiao Yang, Weiqing Liu, Yingce Xia, and Jiang Bian. DDG-DA: Data Distribution Gener- ation for Predictable Concept Drift Adaptation. Proceedings of the AAAI Conference Artificial Intelligence, 36(4):4092-4100 June 2022c. ISSN 2374-3468. doi: 1..16009/aaai..3664200000 URL http::ttt///omm /ojs. aaaí paaiiooe org/ /oeg/indes wnw.. www.ndxe.ppppppe AAAI wtw.a//aarile///aaaeaaaaeeeeaeee wtt.tiiceeeeveee/222/2//2/ Number: 4.
Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. FedBN: Federated Learning on Non-IID Features via Local Batch Normalization, May 2021b.
13

---

Published as a conference paper at ICLR 2026
Haoyuan Liang, Xinyu Zhang, Shilei Cao, Guowen Li, and Juepeng Zheng. TTA-FedDG: Leveraging Test-Time Adaptation to Address Federated Domain Generalization. Proceedings of the AAA! Conference on Artificial Intelligence, (0(18)18858818888 April 2025. ISSN 2374-3468. doi: 10.1609/aai. v39118.344 8..44553.
Xinting Liao, Weiming Liu, Pengyang Zhou, Fengyuan Yu, Jiahe Xu, Jun Wang, Wenjie Wang. Chaochao Chen, and Xiaolin Zheng. Foogd: Federated collaboration for both out-of- distribution generalization and detection. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet. Tomczak, and C. Zhang (eds.), Advances Neural Information Processing Systems, volume 37. pp. 132908-132945, Curran Associates, Inc., 2024. doi: 10.52202/ 079017-4225. URL https://proceeddngg
Guodong Long, Ming Xie, Tao Shen, Tianyi Zhou, Xianzhi Wang. and Jing Jiang. Multi-center federated learning: Clients clustering for better personalization, World Wide Web, 26(1):481-500 January 2023. ISSN 1573-1413. doi: 10.1007/s1 2200-022-01046-x.
Jie Lu, Anjin Liu, Fan Dong, Feng Gu, João Gama, and Guangquan Zhang. Learning under Concept Drift: A Review. IEEE Transactions on Knowledge and Data Engineering. 31(12):2346-233: December a.. ISSN 1558-2191. doi: 10. 1..1099TKDE.200 201.DD....11..88 8.2876857.
Jie Lu, Anjin Liu, Fan Dong, Feng Gu, João Gama, and Guangquan Zhang. Learning under Concept Drift: A Review, IEEE Trunsactions on Knowledge and Data Engineering, 31(12):2346- 2363, December 2019b. ISSN 1558-2191. doi: 10. 10.1109/TKDE.20 201. 201....688188877877777 URL https: //ieeexplore. leee. org/ absttact, poarratt//oooo /wtp///occcc//c/eoceeoooo Transactions Knowledge and Data Engineering.
Conference Name: IEEE
Othmane Marfoq, Giovanni Neglia, Aurélien Bellet, Laetitia Kameni, and Richard Vidal. Federated Multi-Task Learning under a Mixture of Distributions. In Advances in Neural Information Processing Systems, volume 34. pp. 15434-15447, Curran Associates, Inc., 2021.
Othmane Marfoq, Giovanni Neglia, Richard Vidal, and Laetitia Kameni. Personalized Federated Learning through Local Memorization. In Proceedings of the 39th International Conference on Machine Learning, pp. 15070-15092. PMLR, June 2022.
Brendan McMahan, Fider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera Arcas. Communication- Efficcit Learning of Deep Networks from Decentralized Data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, pp. 1273-1282. PMLR, April 2017.
Mahdi Morafah, Saced Vahidian, Weijia Wang, and Bill Lin. FLIS: Clustered Federated Learning Via Inference Similarity for Non-IID Data Distribution. IEEE Open Journal of the Computer Society, 4:109-120, 2023. ISSN 2644-1268. doi: 10.1 10.1109/01CSS.22233222220
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zachary Vetoto, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, Nigh-Performance Deep Learning Library. Advances in Neural Information Processing Systems, 32, 2019.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, Seikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12:2825-2830, 2011.
Shi Pu, Alex Olshevsky, and Ioannis Ch. Paschalidis. Asymptotic Network Independence in Dis- tributed Stochastic Optimization for Machine Learning: Examining Distributed and Centralized Stochastic Gradient Descent. IEEE Signal Processing Magazine, 17(3):14-122, May 2020. ISSN
1558-0792. doi: 10. 1091MS..2000000 2975212.
Rakibul Hasan Rajib, Akil Raihan Iftee, Mir Sazzat Hossain, Sajib Mistry, M Ashraful Amin, and Amin Ahsan Ali. FedCTTA: A Collaborative Approach to Continual Test- Time Adaptation in Federated Learning.
14

---

Published as a conference paper at ICLR 2026
Protection Regulation. Regulation (EU) 2016/679 of the European Parliament and of the Council. Regulation (eu), 679:2016, 2016.
Hanchi Ren, Jingjing Deng, and Xianghua Xie. GRNN: Generative Regression Neural Network-A Data Leakage Attack for Federated Learning ACM Trans. Intell. Syst. Technol., 13(44:65:1-65:24, May 2022. ISSN 2157-6904. doi: 10.1 14.1145/500032
Jorge Reyes-Ortiz, Davide Anguita, Alessandro Ghio, Luca Oneto, and Xavier Parra. Human Activity Recognition Using Smartphones. UCI Machine Learning Repository, 2013. DOL:
Felix Sattler, Claus-Robert Müller, and Wojeiech Samek. Clustered Federated Learning: Model-
Agnostic Distributed Multitask Optimization Under Privacy Constraints. IEEE Transactions on Neural Networks and Learning Systems, 32(8):3710-3722 August 2021. ISSN 2162-2388, doi:
10.1 10.11009/
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership Inference Attacks Against Machine Learning Models. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3-18, May 2017. doi: 10.1 109/P..007..41.
Sebastian U. Stich. Local SGD Converges Fast and Communicates Little, May 2019.
Canh T. Dinh, Nguyen Tran, and Josh Nguyen. Personalized Federated Learning with Moreau En- velopes. In Advances in Neural Information Processing Systems, volume 33, pp. 21394-21405. Cur- ran Associates, Inc., 2020. URL htts: /ttp::::////ccceeee http://ooceeddeioo ngs neurips neurin wwcpppppppaieeeeeeeeeeeeeeeeeeee
paper/2020 hww.aaaaaaaaaaaaa http:/4f111131 http:///hn.....eeeeeeeeeeeeeeeee
-atoo-bbbtatt...aeeeeeeaeeaeece. html.
Ashraf Tahmasbi, Ellango Jothimurugesan, Srikanta Tirthapura, and Phillip B. Gibbons. DriftSurf: Stable-State RReactive-State Learning under Concept Drift. In Proceedings of the 38th International Conference on Machine Learning, pp. 10054-10064. PMLR, July 202 la.
Ashraf Tahmasbi, Ellango Jothimurugesan, Srikanta Tirthapura, and Phillip B. Gibbons. DriftSurf: Stable-State Reactive- Learning under Concept Drift. In Proceedings of the 38th International Conference on Machine Learning, pp. 10054-10064. PMLR, July 2021b. URL https://
proceedings mlr. prwsseresssseeeee /v19 http///1tttttaillleaa
html. ISSN: 2640-3498.
Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. Towards Personalized Federated Learning. IEEE Transactions on Neural Networks and Learning Systems, 34(12):9587-9603, December 2023a. ISSN 2162-2388. doi: 10.1 1091 2022..3322333... 160699. URL https: //ieexxpoxe.. leee. wwt.:/ogg//bbbae http://aattt/ooommn////9//335//
Yue Tan, Guodong Long, LU LIU, Tianyi Zhou, Qinghua Lu, Jing Jiang, and Chengqi Zhang. Fedproto: Federated prototype learning across heterogeneous clients. Proceedings of the AAAI Conference on Artificial Intelligence, 36(8):8432 8440, Jun. 2022. doi: 10.1609/aai.. v36i8................... 20819.
URL https: / /ojs stsijsaaaa. aaai. aaai. org/ iidexx.nxx..///m// php/AAa. http://tticleeeeeeeeeeeeeeeeeeee 20819.
Yue Tan, Chen Chen, Weiming Zhuang, Xin Dong, Lingjuan Lyu, and Guodong Long. Is Heterogene- ity Notorious? Taming Heterogeneity to Handle Test-Time Shift Federated Learning. Advances in Neural Information Processing Systems, (6:27167-27788) December 2023b.
Yue Tan, Chen Chen, Weiming Zhuang, Xin Dong, Lingjuan Lyu, and Guodong Long. Heterogene- ity Notorious? Taming Heterogeneity to Handle Test-Time Shift in Federated Learning. Advances in Neural Information Processing Systems, 36:27167-2711 December 2023c.
U.S. Congress. Health Insurance Portability and Accountability Act 1996. https://wwwt hhs. gov/hipaa/, 1996. Public Law 104-191.
Hemanth Venkateswara, Jose Eusebio. Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5018-5027, 2017.
Cédric Villani et al. Optimal transport: old and new, volume 338 of Grundlehren der Mathematischen Wissenschaften. Springer, 2008.
15

---

Published as a conference paper at ICLR 2026
Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K. Leung, Christian Makaya, Ting He, and Kevin Chan. Adaptive Federated Learning in Resource Constrained Edge Computing Sys- tems. IEEE Joumnal Selected Areas in Commumications, 37(6):1205-1221, June 2019. ISSN 1558-0008. doi: 10. (2.1109//SAC.201992259
URL https::///// http://eeeeeeprreeeee iere..reee.oro. weeeeeeeeooeeeee..rrrr........ org/
abstract//occmmntttt /6646
Michael L. Waskom. seaborn: statistical data visualization. Journal of Open Source Software, 6(60): 3021, 2021. doi: 10.21105/joss.000000 URL https: http://t....r........ 21105/ joss. .essss.03000 03021.
Wes McKinney. Data Structures for Statistical Computing in Python. In Stéfan van der Walt and Jarrod Millman (eds.), Proceedings of the 9th Python Science Conference, pp. 56 61, 2010.
doi: 10.25080/Majora weeeee------oo-0-e
Geming Xia, Jian Chen, Chaodong Yu, and Jun Ma. Poisoning Attacks in Federated Learning: A Survey. IEEE Access, 11:10708-10722, 2023. ISSN 2169-3536. doi: 10.1109/ACCESS.200 3238823.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-MNISH Novel Image Dataset for Bench- marking Machine Learning Algorithms, September 2017. URL 1708.077 07747. arXiv: 100..07777777 [cs].
org/abs/
Zhiqin Yang. Yonggang Zhang, Chenxin Li, Yiu ming Cheung, Bo Han, and Yixuan Yuan. Fedgps: Statistical rectification against data heterogeneity in federated learning, 2025. URL https:
org/ abs 0bbs/2100 2510 20250.
Hongxu Yin, Arun Mallya, Arash Vahdat, Jose M. Alvarez, Jan Kautz, and Pavlo Molchanov. See Through Gradients: Image Batch Recovery via GradInversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 16337-16346, 2021.
Oualid Zari, Chuan Xu, and Giovanni Neglia. Efficient passive membership inference attack in federated learning, October 2021.
Michael Zhang, Karan Sapra, Sanja Fidler, Serena Yeung, and Jose M. Alvarez. Personalized Federated Learning with First Order Model Optimization. International Conference Learning Representations, October 2020a.
Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song. The Secret Revealer: Generative Model-Inversioon Attacks Against Deep Neural Networks. In Proceedings of the IEFF/CVF Conference Computer Vision and Pattern Recognition, pp. 253-261, 2020b.
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. iDLG: Improved Deep Leakage from Gradients, January 2020.
Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated Learning with Non-IID Data. 2018. doi: 00.48550/axxiw.1866.0000 URL http://arxiv. org/abs/Dl /806. 00582. 10Xiv:180060000
[cs].
Jiayu Zhou, Jianhui Chen, and Jieping Ye. Clustered Multi-Task Learning Via Alternating Struc- ture Optimization. Advances Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011.
Hangyu Zhu, Jinjin Xu, Shiqing Liu, and Yaochu Jin. Federated learning non-IID data: A survey. Neurocomputing, 465:371-390, November 2021. ISSN 09252312. doi: 1016/ 2itcom.2021.07.000 URL ttttps://linkingee 5092523122101325:1
elsevier com/ retrreeeerieiiiii
Ligeng Zhu, Zhijian Liu, and Song Han. Deep Leakage from Gradients. In Advances in Neural Information Processing Systems, volume 32, Curran Associates, Inc., 2019.
16

---

Published as a conference paper at ICLR 2026
APPENDIX CONTENTS
A Related Approaches to Distribution Shift and Drift in Federated Learning
18
Implementation Details
19
B.1 Algorithm pseudo-code
19
B.2 Code, licenses and hardware
19
B.3 Models and
settings
20
B.4 Distribution profile extraction
21
B.5 Impact of DPE quality
25
C Discussion: Handling Seen-once Unseen Distributions
25
D FEROMA Robustness Against Malicious Clients
26
Test-Time Model Association under P(Y Concept Shift
27
Privacy Implications of Distribution Profile
28
G Quantification of Costs in FEROMA
30
H Convergence Analysis on Stationary Windows
31
I Additional Experiment Results
32
Ll Generating drifting datasets with ANDA, CheXpert and Office-
32
I.2 Scaling the drifting frequency, non-III types, and non-IID levels
34
1.3 Scaling the number of clients
35
1.4 Results real-world datasets
L,5 Comparison with non-FL test-time adaptation approach
43
16 Generalizability DPE time-Series data
43
1.7 FEROMA robustness against unseen out-of-support distributions
44
L8 Ablation study on Monte-Carle subsampling
45
1.9 Ablation study on threshold profile association
45
I.10 Ablation study on distance function
46
Illustrations of the Dynamic Aggregation Strategy
48
K Evaluation with Higher Number of Communication Rounds
51
Large Language Model Usage Disclosure
522
17

---

Published as a conference paper at ICLR 2026
A RELATED APPROACHES TO DISTRIBUTION SHIFT AND DRIFT IN FEDERATED LEARNING
A rich body of research has emerged to mitigate data heterogeneity in FL. Existing approaches can be broadly grouped into three categories: clustered FL (CFL), personalised FL (PFL), or test-time adaptive FL. (TTA-FI.). Below summarise their core ideas and limitations and contrast them with FEROMA.
Clustered Federated Learning (CFL) CFL assumes that the federation comprises M data distri- butions and aims to learn one model per distribution. Two variations are common:
Hard-CFL. Each client is assigned to exactly one cluster Cm S KG based on predefined eriteria or similarity measures, such that clusters are disjoint and collectively exhaustive: U Cm Kr and Cm Cc. 0 for all m &f n. In principle, each cluster Cm contains distinct subset of clients with similar data distributions, enabling the training of specialized models. Algorithms typically alternate between updating cluster models and re-assigning clients, using clustering model parameters (Sattler et al., 2021: Gong et al., 2024) or performance-basee metries (Ghosh et al., 2020; Jothimurugesan et al., 2023; Morafah al., 2023; Cai et al., 2023). Hard assignments simplify optimisation but struggle when distributions overlap, and they generally require prior knowledge of the number of clusters M-which generally unavailable in practice.
Soft-CFL. This approach allows clients to belong to multiple clusters with certain probabilities, accommodating scenarios where client data may exhibit overlapping distributions. Accordingly, each client holds probability vector (*)) R and trains an ensemble of M models; examples include FedEM (Marfoq et al., 2021) and FedRC (Guo et al., 2024). The joint optimisation of K and model parameters {fmm m= is typically tackled via Expectation -axiatiion-Maiizzaaiionn alternating minimisation (Zhou et al., 2011) 1), which incurs additional communication and may suffer from convergence issues. Soft-CFL also requires per-client labeled data to estimate (k)) making cold-start and unseen-client scenarios problematic.
(xx [k=
Personalized Federated Learning (PFL). PFL reframes FL client-centri, optimization problem, where each client & learns its own personalized model f()) that minimizes its local loss. This approach explicitly addresses the presence of heterogeneous non-IID data distributions across clients, rather than enforcing single global model for all participants. Broadly, PFL methods fall into three classes:
Fine-tuning. A global model is first trained and then locally adapted through additional gradient steps or meta-learning techniques to simplify personalization (Chen et al., 2019; Fallah et al., 2020; Jiang et al., 2023). While simple, fine-tuning requires careful hyperparameter selection (e.g. learning rates, number of steps) and sufficient per-client data to avoid overfitting.
Model decoupling. The network is partitioned into shared (global) and private (local) components. A common strategy is to jointly train a shared backbone while equipping it with separate global and personalized heads (Arivazhagan et al., 2019; Deng et al., 2020b; Collins et al., 2021: Jiang Lin, 2022; Marfoq et al., 2022). Others personalize only batch-norm statisties (Li et al., 2021b) or allow heterogeneous encoder architectures (Diao al., 2020). These approaches improve representational capacity at the cost of increased on-device model size and computation.
Regularizution-base These methods introduce a regularization term to balance local and global objectives. For example, popular strategy is to augment each client's loss with a penalty that ties the personalized model (k) to the global model A: mingos [(K) (d(M) +
(g(h) 0|?
where À balances local versus global objectives (Fallah et al., 2020; Li et al., 2021a). Such li-le formulations yield smooth personalization but introduce per-client hyperparameters, and nested optimization loops.
Despite their effectiveness when large amount of labeled data are available, PFI. methods degrade with limited local samples, incur additional client-side compute and memory costs, and- and-being inherently supervised- are ill-suited for unseen, unlabeled clients or distribution drift at test time.
Test-time adaptive federated learning (TTA-FL). TTA-FL addresses deplomepl distribution shift: after global training has concluded, each client adapts the received model to its own (unlabeled) test data. Most methods rely unsupervised objectives such as entropy minimisation (Jiang & Lin,
18

---

Published as a conference paper at ICLR 2026
2022; Bao et al., 2023; Rajib et al.) self-suuervised contrastive losses (Tan al., 2023b; Chen et al., 2022; Liang et al., 2025). Since gradients must be estimated without labels, the optimization landscape is often ill-conditioned (e.g., in the presence of concept shift): entropy minimization can drive the model toward over-confident but incorrect predictions, while contrastive objectives may collapse when test batches are small or unbalanced a common situation in on-device FI.. To mitigate this instability, recent works restrict adaptation to a few parameters (c.g., batch-norm statistics or single gating weight that interpolates between global and personalized heads (Jiang & Lin, 2022)). However, these approaches still require labeled data during training to learn the personalized head, making them unsuitable for unseen and unlabeled clients test time.
Contrast with FEROMA. While PFL, CFL, and TTA-FL each address aspects of client heterogene- ity, FEROMA unifies their strengths in lightweight,
framework:
ys. PFL. Instead of learning separate model (((k) for each iiient-incurring per-client hyperpa- rameters, inner-loop optimization, and fine-tuning overhead (Arivazhagan et al., 2019; Deng et al., 2020b) FFROMA optimizes single model per observed distribution. Each client then receives its best-matching distribution slice based on a DP-protected profile, eliminating the need for client- specific adaptation supervision. In doing so, FEROMA shifts the objective from client-centric to Intrribution-eetrii optimization.
vs. CFL. Unlike hard- soft-CFL methods that assume M disjoint distributions known priori (Guo et al., 2024; Jothimurugesan et al., 2023; Ghosh et al., 2020; Marfoq et al., 2021; Long al., 2023), FEROMA imposes such assumptions. Its distribution profiles adapt continuously to overlapping or drifting distributions and naturally support unseen clients via the label-agnostic component (R2). Moreover, FEROMA avoids the need to train multiple models per client or transmit multiple updates per round, thereby reducing both computation and communication costs compared CFL approaches (see Table 3).
vs. TTA-FL. Rather than relying on costly and unstable unsupervised fine-tuning at test time (e.g.. entropy or contrastive minimisation (Jiang & Lin, 2022; Chen et al., 2022)), FEROMA performs a one-shot profile extraction followed by -iitrridiitiiitto association. This eliminates the risk of overconfident predictions or collapsed representations, incurs negligible overhead (R5), and generalizes seamlessly to unseen, unlabeled clients.
B IMPLEMENTATION DETAILS
B.1 ALGORITHM PSEUDO-CODE
We provide the pseudo-code for the training phase (Algorithm 1) and inference phase (Algorithm 2) of the proposed FEROMA framework. Both algorithms detail the server-side and client-side operations involved at each round. During training. FEROMA extracts distribution profiles from client data, maps them to previous-round profiles, and assigns models accordingly for local updates (Line 6 and 11 in Algorithm I). During inference, FEROMA matches test clients to the closest available model based on distribution profile similarity, without requiring retraining (Line 9 in Algorithm 2).
B.2 CODE, LICENSES AND HARDWARE
Our experiments were implemented using Python 3.12 and open-source libraries including Scikit- learn 1.5 (Pedregosa al., 2011) (BSD license), Flower 1.11 (Beutel al., 2022) (Apache License), and PyTorch 2.4 (Paszke et al., 2019) (BSD license). For visualization, we use Matplodib 3.9 (Hunter, 2007) (BSD license) and Seaborn 0.13 (Waskom, 2021) (BSD license), For data processing, we use Pandas 2.2 (Wes McKinney, 2010) (BSD) license). The datasets used in our experiments-MNNISI (GNU license), FMNIST (MIT license), CIFAR-10, CIFAR- 100, CheXpert (Irvin et al., 2019), and Office-Home-arc are freely available online. To ensure reproducibility, our code, along with detailed instructions for reproducing the experiments, is publicly accessible on GitHub² under the MIT license. We implement publicly available codes for our baselines (except FedAvg). All experiments were conducted on a workstation equipped with four NVIDIA RTX A6000 GPUs (48 GB each), two AMD EPYC 7513 32-Core processors, and 512 GB of RAM.
19

---

Published as a conference paper at ICLR 2026
Algorithm 1 The FEROMA Framework Training Phase
Require: initial set of clients Ko = (1,2...., Ao = Ko. initial global models {0) (k) JAEK, initial (o))....KK))..A
profiles {dl (*)) HEEKT number of rounds R, DPE @w. distance function ((.,)).
1: for = to R do
2: // Server-side
3: K. UPDATECCLIETPPOOI.GGG (# Update of available clients}
A, + CLIENTSELEEC((EE)) (# Sample clients for training round}
5: // Client-side
6:
for each client & E A, in parallel do
7:
dia)) + (xt (()) Def. 3.1}
8: Send de to the server
9:
end for
10: Server-side
each client E Ar
12:
(wn JjEAr= PROFILEMAPPd!" (dEi)ieA--.) (# Eq. equation 3/equation 4]
0!4) EjEAL-1 Pk (b(h.)) 0(2) (# Eq. equation
13:
14:
Send g(*) to client ht
15:
end for
16:
Clienr-side
17:
for each client k E A. in parallel do g(k) 8(4)
19: ((k) LOCALL (k) :9: (1())
20: Send g( the server
21: end for
22: end for
Algorithm 2 The FEROMA Algorithm Inference Phase
Require: of test clients Keen (1,2,..., Kua), last-round client participation An, last-round models
(8(²)) {0(*) hREAn with profiles (dh JREAR. DPE Φss distance function D(:-).
1: Client-side
2: for each client & E Kuea in parallel do
3: d((k) + to (ares) 0) (# (R2))
Send d((h)) the server
5: end for
6: // Server-side
7: [dk") DEEAn GETPRIME((d)) (d))) hEAR)
for each client Krea
9:
(mm Aills)))eAn beenn +- PROFLEMMP(( ((k)
beAn) (# Eq. equation 3}
10:
j arg maXje-RR vi(ks))
11:
0l4) < -006") Best-matching model]
12: Send o/4 to client &
13: end
14: Client-side
15: for each client & Krea in parallel do
16: 9ih) <- o(hh
17: g(a)= = f ((0))
18: end for
B.3 MODELS AND HYPER-PARAMETE, SETTINGS
We employ a 5-fold cross-validation strategy to evaluate the model's performance, using random seeds (from 42 to 46) for all experiments to ensure reproducibility. For datasets MNIST, FMNIST, and CIFAR-10, we use the LeNet-5 (Lecun et al., 1998) as the base model. For datasets CIFAR-100, CheXpert and Office-Home
use the ResNet-9 (He et al., 2016) the base model. Specifically, we use CheXpert-v 1.0-small, a subset from CheXpert, and use the first 120,000 samples for our experiments. We train the model with batch size of 64 for both training and testing. Each client
20

---

Published as a conference paper at ICLR 2026
allocated 20% of their local data for evaluation. The FL process is conducted over 20 communication rounds, with 100% participation rate and each client performing local epochs per round. The learning rate is set to 0.005, and momentum value of 0.9 is applied to optimize the training process.
B.4 DISTRIBUTION PROFILE EXTRACTION
This section details the implementation of the distribution profile extractor (DPE) é, which is used yet consistent manner to capture the local data distribution of each client in a jnsguurementtstifccaaatiin justification that it fulfills across the federation. In addition, we provide requirement-D sagpurremment----ruuieeeee byrreuureement Definition 3.1.
B.4.1 DPE IMPLEMENTATION.
Let the last hidden layer of the global model produce, for client & in round 4, a matrix of latent vectors (k) E R Our extractor maps these latents d-dimensional profile (k)) in four
privacy-preseriin steps:
SI Global alignment (one shot, raw data). Each client computes the clement-wise minimum and maximum of its latents and sends only the two z-dimensional vectors (2z floats) to the server. The server aggregates by coordinate-we wiste minimum and maximum, obtaining global bounds [m_,m" .m=m,, J. and broadcasts these bounds together with the current model weights. No gradients, labels, or raw examples leave the devices.
S2 Shared PCA on synthetic reference points. Using an agreed-upon uppn random seed, every client draws 200 points uniformly in the range (m m=] and fits PCA map g IR - R (with = 10) on this synthetic dataset only. Because both the seed and the data are identical, the resulting linear projector g is the same on every client, ensuring that Euclidean geometry reduced space is comparable across the federation.
S3 Monte-Carlo moment computation. Each client projects its latents 2, g((
(K)
R B
and draws M = independent Bernoulli(y 0.5) masks (see subsection 1.8 for the ablation study on M). For each mask m, it computes (Jtz
(kk,m) ((k.m)) and {MU (A).m) ((k,m)) }
==1' Here, per (k) (k) are the mean and covariance of the reduced latents (approximating the marginal P(X) of client data), and (uu xx()) are corresponding class-conditional moments (approximating P(Y X)). These averaged moments can be written coordinate-win as hy (z,²) .1 M >M h (m)
then averages over to obtain (pz (k) ((()) and {uu
(R) (1) )
where each h, (m) is an unbiased estimate of the corresponding full-sample statistic, Under sub- Gaussianity with proxy variance 72, the variance of this estimator is bounded by 7 (())))))) (see B.4.4 for derivation).
$4 Disferential-prin -privicy sanitization & profile assembly. The concatenated statistics are each per- turbed with an independent Laplace mechanism (8 = 0) (Dwork al., 2006):
T id
Laplace(0, bi), b: A1,//s
where A is the & -s-sesiityy of statistic hç. For mean or standard-deviation coordinate we conservatively bound A <
Range(h;) The released profile is therefore
m(k)
hu , JTJ + + " 7~ Taplace diag(bi........ ba).)...bbb).
Across the pipeline, the only raw. example-level information ever transmitted is the f2-float (i.c., min/max pair from S?), ensuring compliance with FL privacy constraints. Moreover, with 10.0 and typical p(() 300 and Range(h,) < 10, the worst-case variance 2b} < 2.2x 10-5 is negligible relative to inherent data variability, yet it guarantees (s, 0)-DP at the profile level.
3 Because the synthetic points are fully determined by the shared seed and the broadcast bounds [m m + the same projector g could equivalently be fitted once on the server and broadcast to clients. We compute it locally to avoid broadcasting PCA parameters (projection mean, (zl + z) floats) and to simplify updates when latent ranges evolve across rounds; this choice does not affect privacy.
21

---

Published as a conference paper at ICLR 2026
B.4.2 DISTRIBUTION FIDELITY (R1)
To meet the distribution fidelity requirement (RI), we design our DPE that Euclidean distances in the profile space provably track the 2-Wasserstein distance W2 between client data distributions. PCA (see Paragraph B. B.4.1),
After aligning all clients with a globally consistent, each profile concatenates the first two moments of (i) the marginal latent distribution P(X) and (ii) the class-conditional latents {P(X|Y = u)]a=1-
Consider the following assumptions:
(A1) (Lateet Gaussianity) Each latent distribution can be approximated by Gaussian: P; PM N (1,,, M.). This is standard assumption for deep features.
(A2) (Spectral bounds) There exist constants O < Amin S Amax < Do such that the spectra of all covariances lie in Amax]- This is enforced in our case by Step SI of the DPE, which ensures that all latent representations lie within a global bounding box ",=
(A3) (Approximate commutation) Post-PCA, the covariances are (close to) diagonal: we use commutation to obtain tight identities and can otherwise rely standard operator bounds.
We can define the following proposition for marginals, which then extends to class-conditional. This result applies to the noise-free moment map (i.e., Ø); the stochastic DPE output dt concentrates around ø as quantified by (R3).
Proposition 1 (Bi-Lipschitz equivalence W2 for marginals). Define the profile distance for two clients as
2 11m
rell2 ||Σ E2
If (A1)-(A3) hold, thenfor constants
c_ = mi, (2 Amax) have the two-sided bound
=maxf1, (2V Amin)
2² A S (WW Ez)) S A2.
Consequently W2 and A are bi- bipschttt equivalent* on the set of admissible covariances.
Proof. For notational simplicity, we write each client's marginal profile as (()) = [Mxx vec(Ex)] Then the squared & distance between two client profiles is
(| (( 12 || 122 + l|vec((i 22)
For any matrix A, [[vec(A) = 11. With = Σn - Σ2, we obtain Appendix equation 1:
A (] (2) 1| = ||ا| مم 2 + |ز E21
For Gaussians, W2 || μ2 ll B² (2,, En), where 1 (2 Mn) Th + Mg 2(>://² ///3////////////// is the squared Bures distance (Villani et al., 2008). The mean terms coincide; remains to compare B to | - -2
Under (A3) (covariances commute Σ,E, ΣnE, e.g., are diagonal in the shared PCA basis), / =3/""/""",//>>"" = (-)/>>)//
(M /
1/2 = Σ =//²21/2
Plugging this into the Bures formula:
(E = T
(,/// =
Using the identity for the squared Frobenius norm of the difference between two symmetric matrices:
- Tr(E,) Tr(E2) ( (T
- =| ー 1/2
"We use hi-l-ischhtz equivalence in the standard metric-geometry sense: two distances d₁, dg are bi-Lipschitz equivalent set B if there exist constants C+ > O such that for all a, E B, c. di (a, < dz(a, b) < c4 di (a b). In our case, letting A(d(h1) ,dfhta) = [|d(,,)) dftaa) ||2, Proposition establishes constants c_,
such that c- (((p(k)) P p(k2)) S c+ A over the admissible set considered in the proposition.
22

---

Published as a conference paper at ICLR 2026
Non-IID Level
Max
Min
Mean
Non-IID Level
Max
Min
Mean
Std
0.477
0.288
0.074
medium
1.028
0.159
0,479
0.113
medium
0.517
0.009
0.332
0.084
high
1.016
0.136
0.458
0.108
high
0.535
0.007
0.352
0.078
(a) Wasserstein
(b) Jensen-Shannon
Table 4: Absolute error å between profile distance and the true inter-client distance under three non-IID Levels on MNIST under P(Y (X) concept shift.
By the mean value theorem for f(x) v on [Amins Amax), applied entrywise,
Σ1 Σ2|[F r S B(E, E2) 2 Amain
Σn Σ2][s-
2V Amax
Let a = ||m1 /42 1/(4Amin). Then
12. = |E - Σ2 and l E [kmins kimax] with kmin = 1/(44)max) kmax =
min [a b) S a kb S (ax
which yields c2
A
S
W3 S c A with the stated c and C+.
Empirical validation. We validate this guarantee on 44 850 client-rount pairs generated from MNIST under three levels of non-lID concept shift that perturb P(Y X) (low, medium, high;
see Appendix 1.1 for the protocol). Table reports the absolute error |||a( db (k2) ll2
D(P(")) p((x)) For the target Wasserstein metric. the worst-case error never exceeds 1.1. with mean of 0..49-422. 0.111. We also compute the gap w.r.t. the robustness to the choice of D; the maximum JS error is < 0.54 and the mean 0.32 0.08. These results confirm that our extractor satisfies (R1) with & < 1.1 across a broad spectrum of distribution shifts while preserving local-data privacy.
(JS) distance to demonstrate
B.4.3 LABEL. AGNOSTICISM (R2)
In deployed FL system the clients encountered at test-time rarely possess reliable labels. Require- ment (R2) therefore asks for a sub-vector of every profile that can be computed with features only. Our implementation already provides such a component:
Training phase. Step 83 produces both the marginal moments (NL²) moments {(
(R) >i
and the class-conditional
x(() W The resulting profile splits naturally into
(k)
(K)
(K)
(R)
(K) 20
d.
E ERP
d. ((k)) E Rd-p
Test phase (labels unavailable). The client repeats S1-S2 unchanged, then executes the label-free part of S3, yielding only (14b {.)) Reasonably, the conditional distribution | X) cannot be approximated at test time in the absence of labels. These statistics form the sub-vector (((k)
(2x (k)
RP. fully satisfying the formal condition in the main text. Step S4 applies the
same Laplace mechanism noordinate-wisse so d((R) enjoys the same (e, 8-0) cifffrrential-rivvee guarantee as the full profile.
Because the PCA projector is shared (S2) and the noise calibration in S4 is data-independent, Euclidean distances between two label-agnostic profiles, [|d](()>) - d (ka) ||,, remain a meaningful proxy for the marginal Wasserstein distance between P(X) distributions. Consequently, our DPE allows us to match, at test time, unseen and unlabeled clients to the closest marginal distributions fitted during caaning--stisfyaee Requirement (R2).
23

---

Published as a conference paper at ICLR 2026
B.4.4 CONTROLLED STOCHASTICITY (R3)
To thwart exact fingerprinting of a client whose distribution remains static across rounds-which would otherwise cause it to be matched with certainty at each round, potentially suppressing the contribution of other clients with similarly relevant distributions during aggregation the extractor must output similar but not identical profiles for the same input- preserving the true geometry in expectation. We achieve this with two independent randomness sources.
Monte-Carlo subsampling. Given (()) examples, the client draws M 3 independent Bernoulli(y = 0.5) masks and computes the moments of each subsample. Averaging these estimates yields the profile statistic : 1. Σm h(m)) where each ((m) is unbiased for the corresponding full-sample statistic h = Φ(zt (k) 19 ). If every reduced latent coordinate is sub-Gaussian with proxy variance 7², then
M
72
Var(h) S
My
giving data-dependent variance that shrinks both with sample size and with the number of Monte-Carlo replicas.
2. Laplace mechanism (Step S4). The zero-mean noise Laplace(O, b;) adds fixed variance
26 per coordinate. Because the noise is independent of the subsampling, the total covariance diagonal and bounded:
Cov(db
2t )) =
(k)
Myull + 21
+?
La p I²Ia-
Hence H(d(t) Ze 1/ = O
(8) (K)
(k) Vi 1 and Cov( (d( (k)
(K)
1/ (k)) 3 p²I.. This exactly matches
Requirement (R3) with p² Mryor + 21 2b In our experiments (v(R) 300, 72 < 1.0, 6max = 0.003) this gives 0² s 2.2x10 yielding profile distances that are stable across draws yet impossible to replicate perfectly providing the desired controlled stochasticity.
B.4.5 DIFFERENTIAL- GUARANTEE (R4)
Because FEROMA transmits client-side profiles {d. PKEK, Rhiwhh,, by design, capture each client's data distribution an adversary could, in principle, combine them with model updates to mount stronger data reconstruction or membership inference attacks (Shokri et al., 2017: Zari et al., 2021; Li et al., 2022a; Hitaj et al., 2017; Zhu et al., 2019; Yin et al., 2021). For this reason, Require- ment (R4) mandates an (s, -ddifferrnnt privacy guarantee at the sample level. The coordinate-wise Laplace mechanism (implemented in S4) ensures (s, D)- for the entire profile vector de since each coordinate is perturbed with noise scaled to the same e, and sensitivities are computed in the l norm. Thus, for any neighbouring datasets (x, y) and (x', v/) that differ in example, and for any measurable set C IRd,
Pr(p SS S e (2')
with 5 = 0. In our experiments we set = 10.0; with typical client sizes (,()) > 300, the added variance 2(A1, 2.2 x 10-5 is negligible compared to natural data variability, so inter-profile distances remain reliable. Combined with the many-to- nature of de (*0 ((++a) - where infinitely many distinct datasets map to the same profile, making inversion information information-thhorettcaall theoretically impossible-the this DP mechanism bounds any additional leakage to e-limited factor beyond what is already exposed by model parameters, thereby fully satisfying Requirement (R4).
B.4.6 COMPACTNESS (R5)
Requirement (R5) demands that profile extraction adds only marginal computation and communica- tion overhead relative to standard FL. Our implementation meets this target in both aspects.
Computation. The only non-trivial operation is fitting rank-/ PCA PCA 200 synthetic latent vectors of dimension (Step S2). Using an SVD solver, the cost is Omin(sPCA22 ) (sPCA)22)). In practice >> PCA, the second term dominates: (sPCA)22=
24

---

Published as a conference paper at ICLR 2026
200 = 4 x 10tx flottin--point operations. For typical latent sizes (2 E [128, 2048)).
this results in at most 8.2 x 102 FLOPs negligile compared to the cost of a single local training epoch. For reference, single forward pass (no backward, optimization) with our smallest network on MNIST exceeds 6.5 x 10 FLOPs, while the largest network used CIFAR- 100 requires over 6 x 10t FL.OPs per epoch. Moment computation (Step S3) involves simple summations and products, and is therefore negligible, while Laplace noise injection (Step S4) has complexity (dd).
Communication. Each profile transmits d = (1+ x (1+U) floats, i.e. a mean and (diagonal) covariance entry per latent dimension for the marginal distribution plus the same for each of the U classes. With / - 10 this yields d = 220 floats for MNIST (U 10) and d = 2 020 for CIFAR- 100 (U = 100). By contrast, the model updates in our experiments range from 62 006 parameters (MNIST) to 6 775 140 parameters (CIFAR-100) 100). Hence the profile occupies at most 0/| < 3.5 x 10-3 of the uplink payload, comfortably satisfying the compactness criterion d/ [0] 10-2
These results demonstrate that the DPE introduces only minimal overhead while maintaining the communication efficiency expected in Fl., thereby fully satisfying Requirement (R5).
B.5 IMPACT OF DPE QUALITY
FEROMA relies on DPEs to summarize client data distributions, and its effectiveness therefore depends on the quality of the extracted profiles. This dependency is not unique to FEROMA: many personalized clustered FL methods implicitly rely representation quality through gradients, model updates, or learned embeddings. FEROMA makes this dependence explicit through compact distribution profiles, but does not introduce fundamentally new requirement beyond what is already assumed in modern FL systems.
Importantly, FEROMA is designed to remain stable even when the DPE is imperfect noisy. The profiles produced by the DPE are intentionally low-dimensional and capture coarse, distribution- level statistics rather than fine-grained features. As result, aggregation decisions depend only on relative profile distances, not on exact representations. Noise or approximation error therefore affects FEROMA only through these distances. In practice, the thresholding mechanism further filters out uninformative or noisy associations, and stable aggregation strategies emerge as long as relative similarities are approximately preserved. This behavior is confirmed by the sensitivity analysis which shows that moderate degradation in profile quality-e.g. through increased differential privacy noise-leads to only mild performance impact.
While our implementation instantiates the DPE using a few-round warm-up model, FEROMA itself does not require a pretrained encoder. The DPE can be realized using a variety of mechanisms, including lightweight public encoders, simple statistical descriptors computed directly the data (c.g., feature moments or sketching), or any mapping that satisfics Definition 3.1. Morcover, FEROMA does not require profiles to be available from the first training round: early rounds can follow standard FedAvg, with profile-based aggregation activated once a minimally reasonable encoder or statistic becomes available. This design makes FEROMA applicable even in low-resource settings where no strong pretrained model exists.
Overall, FEROMA does not require task-optimal representations. It only assumes that the DPE provides consistent embeddings or statistics that approximately preserve distributional differences across clients. Under this mild assumption, FEROMA remains robust and effective across wide range of heterogeneity settings.
C DISCUSSION: HANDLING SEEN-ONCE OR UNSEEN DISTRIBUTIONS
As illustrated in Figure and Figure 6, there are two drifting scenarios that FEROMA does not explicitly target: (1) Seen-once distributions, which appear during intermediate training rounds but are absent in the final round; and (2) Unseen distributions, which never occur during training but appear during testing. These two conditions may also coexist.
While FEROMA retains only final-round profiles and models by default, it can be extended to address both cases effectively. For seen-once distributions, the server can optionally store their corresponding
25

---

Published as a conference paper at ICLR 2026
profiles and models during training, provided they yield acceptable validation performance. This enables FEROMA to retain models for all distributions encountered during training, not just those from the final round. For entirely unseen distributions, which are fundamentally unpredictable, FEROMA still assigns the closest available model based on distribution profile similarity. Unlike standard baselines (e.g., FedAvg) that rely single global model, FEROMA selects from diverse set of models trained on different distributional modes. This leads to better initial performance and provides stronger starting point for downstream personalization or test-time adaptation. In this sense, FEROMA complements and can be naturally integrated with test-time adaptive FL methods to further improve robustness in dynamic environments.
Empirical validation. Importantly. both seen-once and unseen distribution scenarios are inherently present our experimental setup. Our dynamic experiments with varying drift frequencies (every 2-4 rounds) naturally create seen-once distributions client data evolves over time. Similarly, our test-time evaluation on cold-start and test-only clients directly evaluates performance on unseen distri- butions. The consistent performance gains of FEROMA across all experimental up to percentage points improvement over anselines-demone handles these challenging scenarios in practice. This empirical evidence validates that distribution- profile-base model selection provides robust generalization even when exact distributional matches unavailable.
aittiionn-acceiinnng that our framework effectively
Training rounds
Testing round
Training clients
Cold-start truining clients
Test-only clients
Scen-once distribution
Not shown in final training round
Figure 5: Seen-once distribution in training stage.
Training rounds
Testing round
R-1
Training clients
Cold-start training clients
Test-only clients
Unseen distribution
Figure 6: Unseen distribution in testing stage.
D FEROMA ROBUSTNESS AGAINST MALICIOUS CLIENTS
In this section, we discuss the robustness FEROMA against malicious or dishonest clients that attempt manipulate the training process (Xia al., 2023; Fenoglio al., 2024). While FEROMA
26

---

Published as a conference paper at ICLR 2026
is not explicitly designed as Byzantine-robust FL algorithm, its pooile-based design introduces several natural protections that significantly limit the effectiveness and impact of adversarial behavior.
Threat model. We consider adversarial clients that may (i) arbitrarily manipulate their local training updates, (ii) deliberately distort their extracted distribution profiles, or (iii) attempt infer or target the data distributions of other clients. We do not assume the presence of secure hardware on the client side, but we assume standard authenticated communication with the server, as in most FL systems.
Limited information exposure. In FEROMA, clients do not have access the distribution profiles. similarity weights, or aggregation decisions of other clients. The server never broadcasts the global set of profiles, nor does reveal the weights assigned to a client previous rounds. Profiles are not shared among clients, and only the aggregated model (or assigned model) is returne.. As a result, malicious clients lack the information required to meaningfully infer, target, or impersonate other clients' data distributions,
across multiple rounds.
Minimal influence of manipulated profiles. Even under an unusually strong adversary that can observe all profiles across rounds, deliberately manipulating a profile yields limited influence. Aggre- gation weights in FEROMA are derived from profile distances: profile that deviates substantially from the population receives negligible weight and is removed entirely by the thresholding mecha- nism. Conversely, an adversary attempting to craft profile that mimics a specific target distribution can only affect the corresponding local neighborhood in profile space equivalenn to influencing a small clustered subgroup or, the extreme case, a single personalized branch. The remainder of the system remains unaffected, preventing global poisoning effects.
Lightweight profiles and secure transmission. The above strong adversary scenario full access to all profiles-is highly unlikely in practice. Nevertheless, FEROMA is designed to remain robust under such assumptions. Because distribution profiles are intentionally lightweight, they can be efficiently protected using standard encryption or secure-channel techniques. In contrast to traditional FL., where encrypting full model updates is often prohibitively expensive due to their size and frequency, encrypting FEROMA'S 's compact profiles introduces negligible computational and communication overhead. This makes secure transmission practical and substantially reduces the likelihood of profile compromise.
Taken together, these properties imply that FEROMA naturally limits the attack surface available to malicious clients. While adversarial manipulation can still affect small subset of closely matched distributions, the system as a whole remains stable and resilient. We view this robustness as a consequence of decoupling model aggregation from explicit client or cluster identities and instead relying on soft, similarity-based associations over compact distribution profiles. Formal defenses against fully Byzantine adversaries are complementary to FEROMA and represent an interesting direction for future work.
E TEST-TIME MODEL ASSOCIATION UNDER P(YIX) CONCEPT SHIFT
In practical settings, class-conditional distributions P(X Y y) cannot be estimated at test time, as labels are typically unavailable. As a result, detecting P(Y X)) concept shift during inference is known impossibility: different label distributions can induce identical feature distributions, leaving no observable signal for detecting the shift based solely on input features. In real-world applications, P(Y X) shifts are often linked evolving user preferences, and the only feasible solution is to query small number of labeled examples at test time to infer the underlying preference.
To assess the feasibility of this approach, we evaluated FEROMA under varying amounts of labeled test samples. Specifically, we measured test-time model association accuracy on MNIST with P(Y X) shifts of increasing severity (low, medium, high non-IID), using between and 50 labeled samples per class. Importantly, these labels were used solely for model selection and not for any model retraining. Results are reported in Figure 7. As expected, increasing the number of labeled samples improves the association quality, approaching the upper bound defined by the optimal association (shown as dashed lines). Moreover, stronger non-IID levels require more labeled examples to achieve a good association, reflecting the inherent difficulty of P(Y | X) shift scenarios.
27

---

Published as a conference paper at ICLR 2026
Mixed
Label
Mixed
90
90
900
Aceurncy
80
80
50
60
60
Revolabeh Aumociaiin
Foy-
Koone Asecinatoon
Kaonn Anaiiiakos
Krown Pusen
50
50
50
Laheled samples
Labeled samples
Labeled samples
Figure 7: Test accuracy of FEROMA on MNIST under P(Y X) concept shift, as a function of the number of labeled test samples per class used for model association. Dashed lines indicate the performance under (optimal) known model assignment. Results are shown for three levels of non-IID severity.
Practical instantiation. From these results, we conclude that using 20 labeled samples per class strikes a good trade-off between performance and labeling cost. Accordingly, we adopt this configura- tion in all our experiments involving test-time association with FEROMA. To ensure a fair comparison. baseline methods are evaluated under (optimal) known association, since they do not natively address P(Y X) shifts or propose alternative strategies.
Numerical experiments. For clarity and completeness, we additionally tracked the performance of FEROMA under the (optimal) known association condition across all experiments. This comparison isolates the influence of test-time association errors from the effects of distribution drift and shift occurring during training. Tables 5-8 report the test accuracy of FEROMA when using either the known association the automatically detected association (with 20 labeled test samples per class) across four datasets: MNIST, FMNIST, CIFAR-10, and CIFAR- 100. As expected, the results show that under higher non-IlD severity, the gap between known and detected association widens, reflecting the increasing difficulty of matching clients to appropriate models in the presence of stronger P(Y|X) shifts. Nevertheless, FEROMA maintains strong performance even when relying on few labeled samples, demonstrating robustmess to moderate test-time association errors.
In addition, these results confirm that when labels are available-as is the case during train- ing FER-FERRMA consistently maintains high performance across all levels of non-lID severity and client drift rates. This stability highlights the robustness the framework in handling both distri- bution shift and drift during training, demonstrating that FEROMA's 's adaptive aggregation remains effective even under highly heterogeneous and dynamic conditions.
Non-IID Level
Medium
High
Drifting
5/20
10/20
20/20
5/200
20
Known Association
89 58 + 0.50
89.48 + 0.45
89.80 0.75
88.90 0.53
88.87 + 0.68
89
0.64
88.15 + 0.66
88.66 + 0.51 88.41 0.68
samples
78.15
80.99
80.33
78.38
Table 5: Test accuracy of FEROMA MNIST under known test-time model association and automatic detection with few labeled samples (20 per class). Results are reported across low, medium, and high non-IID levels, with varying numbers of drifting clients.
F PRIVACY IMPLICATIONS OF DISTRIBUTION PROFILE
Besides model parameters, FEROMA also transmits the client-side profiles ) Because, by design, profile distances approximate distribution divergences (requirement R1 in 3.1), an nonest-but-curion server (or external cavesdropper) could leverage d(a) together with model updates to mount stronger reconstrconttttiio or membership- inference attacks (e.g. (Shokri et al., 2017; Zari al., 2021; Li al., 2022a; Hitaj et al., 2017; Zhang al., 2020b; Ren et al., 2022;
JEEK Rdd
28

---

Published as a conference paper at ICLR 2026
Non-IID Level
Medium
High
Drifting
5/20
10/20
20/20
5/220
10/20
Known Association
75.37 0.33
75.23 a 0.42
75.25
0.27
74,83 a 0.38
74.64 0.37
74,60
0.36
74.29
0.57
73.83
0.0.2
73.93 0.36
samples
72.49
70.62
72.48
67.14
1.40
Table 6: Test accuracy of FEROMA FMNIST under known test-time model association and automatic detection with few labeled samples (20 per class). Results are reported across low, medium, and high non-IlD levels, with varying numbers of drifting clients.
Non-IID
High
Drifting
5/20
10/20
20/20
5/20
10/20
20 20
5/20
10/ 20
20/22
Known Association
42.22 0.54
42:47 a 0.32
4.314
4.78
0.25
41.47
0:48
40.76
41.01
0.50
40.74 0.61
Few labeled samples
37.19 0.76
36.61 + 1.61
37.37 1,53
35.23
0.45
33.48
1.04
34.04 081
31,96
1.04
29.89 1.42
29.87 1.09
Table 7: Test accuracy of FEROMA on CIFAR-10 under known test-time model association and automatic detection with few labeled samples (20 per class). Results are reported across low, medium, and high non-lID levels, with varying numbers of drifting clients.
Zhu et al., 2019; Zhao al., 2020; Yin al., 2021)). Accordingly, we treat profile transmission as an additional leakage channel and explicitly constrain it via multi-layer protection mechanism: (i) structural compression flow-dimensional, many-to-one profiles), (ii) intentional stochasticity in extraction to reduce linkability across rounds, and (iii) formal (c, 5)-DP guarantee at the sample level for each released profile.
For this reason, it is necessary to satisfy requirement R4, which provides the formal DP guarantee for profile release. Requirement R4 (see subsection 3.1) endows the Distribution- Profile Extractor do (R=(+)x= + Rd with (e, 8)-differential privacy at the sample level: for any neighbouring datasets (x, y) and (x', y²) that differ in one example and every measurable S TRd.
Pr(o@(1,)) v) E S] < e Pr(pp(t', v') E S +
First, Φu intentionally many-to-one and low-dimensional: aggregates statistics (e.g.. moments in latent space) into vector in Rd with d & the input data dimension. As consequence, infinitely many distinct datasets can induce the same profile, so the mapping is not uniquely invertible in general and does not preserve sample-level detail. This structural compression does not, by itself, constitute formal DP guarantee, but it reduces the granularity of information exposed and makes exact reconstruction highly underdetermined without strong auxiliary assumptions. Second, we enforce (e, 8D at the sample level for each released profile, which bounds the incremental leakage attributable to profile transmission (beyond what may already be exposed by model updates).
Numerical experiments. We evaluated the impact of enforcing DP FEROMA under varying privacy budgets e, focusing on the privacy-utiliy utillty trade-ofi off in realistic FL scenarios. As shown in Tables 9 and 10, we tested FEROMA across all four types of statistical heterogeneity-P(()) P(Y). x), and P(X Y) at three levels of non-lll) severity (low, medium, high). Overall results are summarized in Figure We observe that FEROMA maintains robust accuracy for moderate budgets commonly targeted in practice (5 I 10), achieving performance comparable to the non-private (No-DP) baseline. In contrast, for very strict budgets (e {1, 2}),, the injected noise is intentionally
Non-IDD Level
Low
Medium
High
5/20
29/20
5/20
10/ 201
20/
5/20
201/20
Known Association
38.88 = 0.28
39.62 + 0.50
39.36
0.30
35.63 0.30
35.71 + 0.29
35.72 =
0.08
31.75
0.14
32.15 0125
labeled samples
0.50
30.05
1
Table 8: Test accuracy of FEROMA CIFAR-100 100 under known test-time model association and automatic detection with few labeled samples (20 per class). Results are reported across low, medium, and high non-IID levels, with varying numbers of drifting clients.
29

---

Published as a conference paper at ICLR 2026
85
Accuracy
80
75
Low Non IID Level
Medium Nee-IID Lovel
High
10
No DP
Differential Privacy (E)
Figure 8: Average test accuracy of FEROMA across different privacy budgets (c) and non-IID types. Results are aggregated over P(X), P(Y), P(Y)X), and P(XJY) scenarios.
large and we observe accuracy drops, especially under high non-IID severity a behavior that is expected under strong privacy constraints. These results indicate that DP-protected distribution profiles can provide formal privacy guarantees with minimal utility loss at moderate e, supporting deployments subject to regulatory constraints such GDPR (Regulation, 2016) or HIPAA (U.S. Congress, 1996).
Non-IID Type
P(X)
P(Y)
Non-IID Level
Low
Medium
High
Medium
High
1
77.39 a 6.23
77.78 a 11.08
57.83 a 13.07
98.48 a 0.73
96.94 a 3.06
97.72 1.99
2
86.64 + 2.10
83.32
69.88 11.25
98.25 1.14
96.98 + 2.98
97.21 = 2.99
89.00 + 0.38
85,15
84.71 1,71
98.69 0.339
97.43 2.85
97.21 = 2,93
10
88.72 0.50
86.46 2.45
85.69 0.43
98.67 0.44
97.86 2.02
97.29 : 3.05
No DP
88.67 0.30
86.50 2.48
85.81
0.36
98.66 0.44
97.94 2.01
97.24 2.96
Table 9: Test accuracy of FEROMA under different privacy budgets e for non-lID types based on feature distribution skew P(X) and label distribution skew P(Y) on the MNIST dataset. Results are reported for low, medium, and high non-Ill) levels.
Non-IID Type
P(Y())
P(X
Non-IID Level
Medium
High
Low
Medium
High
79.15 3.25
74.23 3.20
71.63 4.94
83.95 5.14
77.45 9:22
67.14 7.73
3.44
75.10
4.03
77.44
6
84.50 2.22
78.23 + 3.43
78.32 2.08
1.59
87.75 + 1.99
86.28 7.10
= 10
= 2.63
79.16 2.74
78.02 2.86
89.27 + 0.87
89.63 + 1.13
86.96 2.58
DP
85.04 2.44
80.09 + 2.14
78.38 + 2.62
88.81 + 1.94
89.02 + 1.59
88.29 a 1.12
Table 10: Test accuracy of FEROMA under different privacy budgets for non-III types based concept shifts P(Y and PYX on the MNIST dataset. Results are reported for low, medium, and high non-IID levels.
G QUANTIFICATION OF COSTS IN FEROMA
In this section, we quantify the communication and computation overhead introduced by FEROMA and compare it explicitly with standard FedAvg. While FEROMA introduces additional operations for extracting and transmitting distribution profiles, these costs are intentionally designed to be lightweight and independent of model size.
30

---

Published as a conference paper at ICLR 2026
Communication cost. On the communication side, FEROMA transmits exactly the same model parameters as FedAvg, plus a single low wwwddimensional distribution profile per client per round. The size of this profile depends only the descriptor dimension and the number of labels, and is independent of the underlying model size. As a result, the relative communication overhead decreases rapidly as models scale. Table 11 reports the per-client upload/download size and corresponding transmission time under bandwidth of 20 MB/s for representative models of increasing scale. Across all settings, the additional communication overhead introduced by FEROMA remains below 0.4%, and becomes negligible (below 0.01% for large models.
Datuset/ / Model
Method
Upload Download
Time (20MBA)
Overhead
MNIST LoNet5 (62K)
FedAvg
PEROMA
248.85 88 KB
0.488
+0.35%
CIFAR-100 Re-Ner- (1.65)
FadAvg
13,000
HEROMA
6.61
18221s
+0.15%
CIFAR-100 VIT-B/16 (~86M)
FadAAg
344.0
FEROMA
344.01
688.016
0.01%
Table 11: Per-round communication cost per client for FedAvg and FEROMA under different model scales.
Computation cost. On the computation side, the only non-trivial additional operation introduced by FEROMA is the computation of rank-1 PCA over PCA 200 synthetic latent vectors of dimension during distribution profile extraction (Step S2). This operation costs at most approximately 8.2 x 10? FLOPs for typical latent dimensions (z [128, 2048).. For comparison, even a single forward pass of the smallest MNIST model already exceeds 6.5 x 10 FLOPs, while training the largest CIFAR-100 model requires over 6 x 10t FLOPs per epoch. The remaining operations in the DPE, including moment computation and noise addition, are linear in the descriptor dimension and therefore negligible.
Overall, FEROMA introduces well below 1% additional computation relative to standard local training, while providing substantial robustness benefits under distribution shift and drift.
H CONVERGENCE ANALYSIS ON STATIONARY WINDOWS
This section addresses the convergence behavior of FEROMA. Under unbounded and potentially adversarial concept drift, classical global convergence guarantees are in general unattainable, since the underlying objectives may change arbitrarily over time. We therefore analyze FEROMA stationary time window between two distribution-shife events, where local data distributions are approximately constant. This "piecewise-ssationary under perpetual drift are not meaningful. For this analysis, we adopt the common idcalized setting of a fixed set of clients with full participation in every round, although FEROMA supports partial participation in practice.
viewpoint is standard when formal guarantees
Stationary window. Fix contiguous interval of communication rounds T (to..... +T T during which each client distribution stationary:
p(k)
VIET, ke k K}.
For each client k; define the local objective
(x) = B(zzg) pk) [E(O; v)],
and the weighted global objective F(0) : =K k= (0), where pk 0 and Σx Pk 1. Within T. the local training step in FEROMA corresponds to standard SGD on fi. with fixed number 2 of local steps per communication round.
We adopt standard conditions from local-SGD/ decentralized SGD analyses (c.g., (Koloskova et al., 2020; Stich, 2019; Gao al., 2021; Pu al., 2020)).
31

---

Published as a conference paper at ICLR 2026
Assumption A1 (Smoothness and bounded variance). For each k, fx is L-smooth and lower bounded by f Stochastic gradients 9ith local step h E {0....: H 1} are unbiased and have bounded variance:
(()) 1.4
E 11ghhh
(sx( so¹
9thh
for all / T and k (I....,K K}.
Assumption A2 (Profile-inuuced mixing matrices). At each round E T, FEROMA induces a mixing matrix W. E RK*K with entries (W,)aj := wt (k.)) where wi (kj) denotes the (possibly thresholded and renormalized) association weight used in Eq. (5). We assume:
Symmetry and double stochasticity: W, W.W. W,111,
1 1
Expected consensus rate (time-varying topology): there exist constants 8 E (0, 1] and an integer block length B 2 1 such that, for all X E <R and all integers
E ||
S (1 -
where WeB == W(LHIBB-'' & X 11 and the expectation is over the randomness of (WIB.... ((e-1)B-1)-
Assumption A2 is standard in decentralized SGD with changing topology and implies that repeated mixing contracts disagreement towards the uniform average with factor (1 - (3) (Koloskova et al., 2020; Pu et al., 2020). In FEROMA, such mixing arises by building similarity weights from distribution profiles and applying a normalization step that yields (approximately) doubly-stochastic matrices. Moreover, given requirements (RI) and (R3), profile distances track underlying distribution distances up to small distortion and the injected profile noise has bounded covariance; under a stationary window, this makes it natural to assume that the resulting similarity- -based matching can be chosen so that the induced (We} satisfy the expected consensus condition in Assumption A2 (cf. Koloskova al. (2020)).
From FEROMA decentralized local-SGD. Let θ := . E R denote the matrix of client parameters at the beginning of round t. Over stationary window, one round of FEROMA can be written the generic decentralized local-SGD form
0 O,W, - 7.Gr,
where O,W; We is the profile-based aggregation (Eq, (5)) and G; stacks the stochastic gradients accumu- lated during the H local SGD steps starting from the mixed parameters. Concretely, the k-th column of O,W, equals Σ; Wi (k-j) o(3), matching Eq. (5). This recursion coincides with decentralizee/lloaal with time- varying mixing matrices studied in Koloskova et al. (2020), specialized to a star-shaped implementation where the server computes W; and broadcasts the mixed models, but the update is applied client-side.
Implication (convergence within 7). Under Assumptions A1-A2 and on stationary window T. FEROMA is a special case of decentralized local-SGD with changing topology. Therefore, existing results for Eq. equation 3 apply directly, yielding convergence guarantees F (for non-convex objectives) with an optimization error term and an additional consensus term that depends the mixing quality (through (8), the number of local steps H, and gradient noise o² (see, e.g., (Koloskova et al., 2020)). Intuitively, better profile-inducad mixing (larger ()) reduces client disagreement and improves the rate, whereas very strict privacy/noise settings that perturb profile similarities can degrade 3 and thus slow convergence. Formally characterizing how DPE hyperparameters affect the spectral gap of W, is an interesting direction for future work.
stationary point of
I ADDITIONAL EXPERIMENT RESULTS
I.1 GENERATING DRIFTING DATASETS WITH ANDA, CHEXPERT AND OFFICE-HOME:
For datasets MNIST, FMNIST, CIFAR-10, and CIFAR- 100, we generate drifting datasets across clients with four non-IIID types using ANDA. ANDA (A Non-IID Data generator supporting Any
32

---

Published as a conference paper at ICLR 2026
Training rounds
Testing round
R-1
R
Sample client
Color Cooen
Color Red Romskon Degree 270"
Cofor 10k
Coke Red Ratakko Degiee
Color Bike
Rocaton Degree
Rosaicn Degano 180"
Rasaionn Degiee 180"
Figure 9: Distribution drifting in P(X) with MNIST.
Training
Testing round
2
R-1
R
Sample client
Labelc
Labalc
Labde
Labik (1,15.5)
IV.
Figure 10: Distribution drifting in P(Y) with MNIST.
kind) is a toolkit designed create non-IlID datasets for reproducible FL experiments. supports datasets MNIST, EMNIST, FMNIST, CIFAR-10, and CIFAR-100, and facilitates five types of data distribution shifts:
Feature distribution skew (covariate shift): Marginal distributions P(X) vary across clients.
Label distribution skew (prior probability shift): Marginal distributions P(Y) vary across clients.
Concept shift (same X, different Y): Conditional distributions P(Y vary across clients.
Concept shift (same Y, different X): Conditional distributions P(X vary across clients.
Quantity shift: The amount of data vary across clients.
ANDA enables the generation only shifting datasets or shifting with drifting datasets, allowing clients to possess datasets with varying distributions across training rounds or test round.
ANDA applies commonly used approaches (Sattler er al., 2021; Deng et al., 2020b; Guo et al., 2024; Jothimurugesan al., 2023; Ghosh al., 2020; Marfoq al., 2021; Long al., 2023; T. Dinh et al., 2020) to generate drifting heterogencous datasets:
P(X) (Figure 9): Each image undergoes one of three color transformations (blue, green, or red) and one of four rotations (0", 90", 180", or 270"), with distinct distributions applied to each subset. P(Y) (Figure 10): Each client receives data only from certain classes. For example, in the MNIST dataset, the dataset in training round 2 only has digits 2, 4, and 7, while the dataset in training round 3 has images of digits 1, and
X)Y (Figure 11): Given identical feature distributions (e.g., image pixels). labels differ between clients. For instance, the MNIST dataset, the dataset in training round labels the digit '8' as '8', ".' as '5', and '5' '1', whereas the dataset in training round 3 labels '8" 'I', 'I' '5' and '5" as '8"
PYX (Figure 12): For the same label, different features are applied. For example, in the MNIST dataset, the dataset in training round applies a blue hue to images labeled '0', while the dataset in training round 3 applies red hue the same label.
For the roaldworld dataset CheXpert, we do not apply any image augmentation or label modification in order to preserve the correctness and integrity of the data. To simulate different levels of data
33

---

Published as a conference paper at ICLR 2026
Training
Testing round
2
4
R-1
R
Sample client
2
Label SHa0ing: 3-111
Swapping: 115, 51-81
SwApping T1-8, 51-51
3-55
Figure 11: Distribution drifting in X( with MNIST.
Training rounds
Testing round
Sample client
5
G
6 5 3
8
Lirated Cake [ViSkee, 1% Grean W:: Blac]
Linggedd Cake: (V 1'ite, Wiited)
Togetel Color IV. Red, 1'Blac, 9:Bbao)
Vargoed Cake:
Targered Color Rod. Stoe 11 Mlac))
[V-CConn, 'T: Ciress 11: Grown
Figure 12: Distribution drifting P(X with MNIST.
heterogeneity, we partition the dataset into multiple distributions based on three metadata attributes available in the dataset: ViewPosition, Age, and Sex. Specifically, the dataset is split according to the following criteria for each non-IID onD level used in our experiments:
Low Non-lID Level: distributions based on ViewPosition (Frontal / Lateral).
Medium Non-IID Level: 4 distributions based on ViewPosition (Frontal Lateral) and Age (250/
<50).
High Non-IID Level: distributions based on ViewPosition (Frontal / Lateral), Age (250 / <50), and Sex (Male / Female).
For the real-world dataset Office-Home, we do not apply any image augmentation or label modifi- cation; instead, we directly the dataset in its original form. For experiments, we restrict the dataset to images with labels from 0 to 9. To model different levels of data heterogeneity, we leverage the four inherent domains of the dataset: Art, Clipart, Product, and Real- World. The domain shift across these categories naturally induces heterogeneity in the data distribution.
I.2 SCALING THE DRIFTING FREQUENCY, NON-IID TYPES, AND NON- NON-IID LEVELS
We evaluate the effects of data heterogeneity using four non-llI dataset types across 20 clients, generated with ANDA at three distinct levels of heterogeneity and three levels of drifting frequency.
Drifting frequency. Client's local data change every round, and we scale the local data distribution drifting frequency. At level one, each client's local training data distribution drifts every four rounds. At level two, each client's 's local training data distribution drifts every two rounds. At level three, each client's local training data distribution drifts every round. The test time distributions are always under drift.
Non-IID Types and Levels. We evaluate FEROMA under four common types of non-IID data distributions and define three levels of heterogeneity (low, medium, and high). The full configurations summarized in Table 12 and detailed as follows:
34

---

Published as a conference paper at ICLR 2026
Feature distribution skew (P(X)): Each client is assigned unique combination of data aug- mentations (c.g.. rotation and color transformation), applied consistently to all local samples. The non-lID level controls the number of available augmentation choices. For example, at the Level Medium, a client may apply one rotation from {0", 180") and one color from (Red, Green, Blue) all images. At Level Low, the "Original" color indicates no color transformation.
Label distribution skew (P(Y)): Each client retains samples from a subset of classes. For MNIST, FMNIST, and CIFAR-IO,, each client holds data from 2 classes; for CIFAR-100, from 20 classes. We define a bank of class subsets (e.g., 4 banks of {[0,4], [1,9], [3,5], [6.9]] at Level Low), and clients randomly sample from these banks. Increasing the number of banks increases the heterogeneity level.
Conditional label skew (P(Y)X)): Clients receive relabeled versions of subset of classes via label permutation. For example, at the medium level, class pool (2, 3, 5, 8) may be randomly permuted to {5. 8, 3. 2), mapping images originally labeled as '2' to label '5'. The number of classes in the permutation pool increases with the non-IID level.
Conditional feature skew (P())))): Clients apply different augmentations to the same class. For instance, Client A may apply 0 rotation to class '5', while Client B applies 180% rotation. Augmentations are limited to rotations ((0", 90,, 180", 270" }) and colors ((Red, Green, Blue)). For MNIST, FMNIST, and CIFAR-IO, this applies to 8 classes; for CIFAR-I00, 80 classes. The level of heterogeneity is scaled in the same way as for P(Y), using banks of class-specific transformations.
non-IID type ((x)
P(Y)
P(
P(X YY)
Low
Rotation f0%, 180%, 270"), ), Color {Original)
ABank
4Swupped Cles = (44)
#Bank
Medium
Routiion (09, 180%) Colar (Red, Bloe, Green)
#Bank 6
eSvapped Class (60)
#Bank -6
High
Rotation f0% 90% 180", 270") ). Color (Red, Rlue, Groen)
ARonk
#Swapped Class (80)
#Bank 8
Table 12: Summary of non-lII data heterogeneity configurations across levels. Each type of heterogeneity corresponds specific distribution shift. For P(Y|X), the numbers swapped class for CIFAR-100 are 40, 60, and 80, accordingly.
We dynamically scale the size of training dataset based on the factor of drifting frequency and number of clients to ensure each subset preserves enough samples to train the local model. Tables 13 to Table 28 present detailed results of FEROMA and all baseline methods across various drifting frequency, non-IlI types and levels. For experiments P(Y]X) concept shift, we keep the test time distribution the same as the last training round (see Appendix E for more details).
Non-IID Level
Low
Mediom
High
4 Drifting
$/20
10/20
20 20
5/20
16/ 20
20
5/20
10/
20/20
FedAug
66.55
67.92
72.12 3.45
73.19
73.66 +
73.25 1.54
54.77
$4.32 2.68
58.54
FeIRC
25.36 1.71
25.79
26.42 1:33
11.35 0.00
11:35 : 0.00
11:35 0.00
11:35
11.35 0.00
RedEM
25.36 1.71
25.79 1.38
2 1:33
11.35 0.30
11.35 = 0.00
11.35 e 0.00
11.35 0.00
11.35 0.00
11.35 0.00
FeSEM
2.33
70.39
6.39
72.97
71.52 4.04
47.95
49.89
CFL
74.28 1.51
75.65 a
78.69
82.95 1.32
80.94 2.30
80.23 3,33
68.03 2.18
68.78 1.94
70.65 4.20
IFCA
38.68 8.39
37.48 11.31
4
40.41 15:49
39.19 4.71
3 5.59
30.00 L1.54
31.27 : 15.08
22.01 14.43
PFodMe
35.20 2.54
43.45
47.72
43.29 4.98
46.52
50.22 3,47
30.31 5.43
9.46
34.76 3.16
APEL
55.29
61.87
3
70.88
71.42
72.26 2.50
55.03 3.60
5:79
61.35
FedDria
41.31 2.08
46.08 8.46
46.41 4.26
50.94 10.99
$4.49 :: 8.04
$9.06 a 4.54
39.63 5.776
39.47 8.12
37.40 5,50
AIP
78.74 2.59
82.43 0.72
84.00 a: 1.53
82.56 1,50
76.29 2.64
80.09 a: 1.75
60.55 a: 14.14
39.00 19.31
51.14 14.01
FEROMA
88.90 01,33
88.67 0.30
89.52 0.42
87.14 2.29
$6.50 = 2.48
86.08 + 3.36
$6.40 0.19
X5.81 0.36
85.74 1.17
Table 13: Performance comparison across three different non-Ill) Levels of P(X) and three distribu- tion drifting levels the MNIST dataset.
I.3 SCALING THE NUMBER OF CLIENTS
We evaluate the scalability and efficiency of the proposed approach across varying numbers of clients MNIST dataset, with the non-llI Level Medium (see subsection I.2 for more details). To ensure each client has sufficient local training data, we adjust the dataset size based on the total number of clients. For 10 clients, reduce the dataset to half the size used in the main experiments
35

---

Published as a conference paper at ICLR 2026
Non-IID Level
Low
Medium
Hligh
Drifting
5/20
10/20
20 20
5/20
10/20
20/ 20
5/300
10/20
20/22
FedAvg
12.47
74.70
FedRC
68.72 12.82
16.24 7:75
8.20
36.44
68.54 12.47
6.85
47.94
13.45
a 14.30
79.22
FedEM
66.81 14.63
78.29 4.300
63.10
48.68
62,98 6.57
73.85 a
44.29
59.28 12:07
78.05 6.27
80.00 7.15
4..17
64.05
75.46 3.73
75.44
10.98
63.99 11:30
7.37
83.78 7.20
9892 #
77.18
86.05 5.89
74.45
83.15 3:74
90.35
IPCA
42.55 12.68
47.09 12,40
47.52 22.21
39.16 15.13
=4.78
23.50 +
22.06 2:74
27. 5.85
pledMi
15.59 11.11
47:25 8.41
42:03 17.83
38.94 13.87
37.17 11.39
25.50 5.17
50.80
28.72 7:36
29.50
APFL
9.55
56.67
RelDinn
54.08 12.62
58.21 8.00
48.36 19.20
43.86 12.37
4 11.85
11.85
33.75 5.75
36.44 6.01
36.60 9:05
42. 36.36
ATP
42,75 25.09
76.00 20.47
95.62 2.20
38.30 + 26.16
87.20 8.07
84.64 13.35
36.23 21.01
83.91
90.27 6.21
FEROMA
96.50 3.23
96.66 0.44
99.36 0.29
97.12 1.75
97.94
98.67 1.08
96.91 1.89
97.24 2.96
99.32
Table 14: Performance comparison across three different non-IID Levels of P(Y) and three distribu- tion drifting levels the MNIST dataset.
Non-IID Level
Low
Medium
High
# Drifting
5/20
10/20
20
5/200
10 20
20/20
5/20
10//20
20
73.51
7291 0.8
72.93
6
2.11
2.04
56.38 1.82
FedRC
29.62 4.95
28.33
8.02
3.99
12.91 2.29
2
12.96 a 3.62
12.12 2.26
11.62
FodEM
29.61 4.944
28.79
28.34
2.29
2.94
12.96 3.61
12.12
11.62.1
CFF.
+
75.01 + 0.79
75.53 0.62
66.82 1.56
67.26 + 1.22
66.31 2.01
58.88 1,40
59.95 1.66
58.11 1.78
IPCA
76.89 2.60
77.15 2:31
78.31 3366
71.64 2.50
09.20 4.04
63.46 3.02
60.51 = 1.13
58.44
pFedMe
0.34
APEL
0.55
9
92.54
92.033
91.64
91.22 0.68
91.05
90.90 0.58
89.87 0.85
FedDrift
91.20
92.55
0.42
92.3 0.27
89.71 3.43
89.71 2.82
90.54 1.03
87.51 2.09
B4:43 a 2.29
83.87
76.94 0.72
75.79
77.10
67.73
68 14
60.14
1.01
PEROMA
89.58 a 0.50
89.48 0.45
89.80 0.75
88.90
0.53
88.87 0.68
89.01 0.64
15.15 0.66
88.06 0.51
88 0.68
Table 15: Performance comparison across three different non-llI Levels of P(Y and three distribution drifting levels on the MNIST dataset.
Non-ID Level
Low
Medium
High
Drifting
10/201
20
10/20
20
5/20
10/20
20 20
FedAvg
73.39 : 9.75
$1.57 2.86
73.25
73.09 4,06
74.97 a 4,10
71.56
74.30 5.09
3.577
2.03
FelFM
39.48 16.08
40.06 + 12.86
30:86 19.18
12.87 3.04
7.99
14.70 + 4.40
12.31 +
12.01
12.55 2.39
FeSEM
70.07 + 7.57
74.38 3.34
74.30 4.84
67.89 9.47
62.18
63.74 = 10.06
62.85
59.89 7.05
56.57 6.46
82.84
79.30
78.22
76.52
77.83
IECA
49.12 13.79
35.88 11.68
33.13 10.31
13.08
30.06
34.99
29.25
PFedlMe
47.31 9.19
54,89 7.12
53.18 à: 7.58
39.63 10:16
42,60 6.62
48.26 7.80
36.97 à: 7.00
37.18
45.86 14.27
APFI.
66.94 5.64
71.04
9.07
65.20
67.60 8.00
62.09
60.84
FedDuit
12.31
5.44
46.14
6.16
$1.62
45.44
ATP
34.74 3.67
79.95 6.33
78.06 12.67
73.81 10.86
78. 94 5.80
78.62 10.91
79.43 4.01
76.84 10.65
79.88 6.86
FEROMA
87.26
88.81 1.94
1.17
88.29
Table 16: Performance comparison across three different non-lID Levels of P(X and three distribution drifting levels on the MNIST dataset.
36

---

Published as a conference paper at ICLR 2026
Non- HID Level
Low
Medium
High
Dritting
5/20
10/20
20/200
5/20
10/20
20
5/20
10/
20/20
FodAvg
21.56
PedRC
25.88 0.24
26.17
2
13,40
12.30
222
10.94 0.73
10.66
11.54
FedEM
25.88 0.26
26.18 0.60
a.7.1
13.39 1 0.85
12.30 1.08
15.55 2:24
10.94
10.66 a 0.31
a
PeSEM
35.03 0.66
35.38
34.87 0.996
21.44 1.93
20.73
4.776
19.28 2.52
19.26 2: 1.55
18.99 1.43
39
16.90
IPCA
51.08 11.08
33.18 5.00
33.75 2.755:2..12
17.33 1.54
20.28 + 2,92
21,82 3.00
18.83 + 3.16
18:45 1.62
18.37 1,40
pPodMo
2.45
2:35
21.51 0.663
16.94 0.47
18.12 0.98
18.66 1.59
16.55 1 1.77
17.41 1.72
15.25
APHL
27.88
31,59
34.07
20044
2.41
2.49
21.10 1.12
FedDrift
3.33
a 4.98
32.25 a 4.15
17.31 1.42
20.50 à 1,97
21.10
17.86 1.98
18.65
17.92 1,92
ATP
53.53 1.17
32.15 3.73
51,
2,78
28.27 1.66
29.93 1,77
28.17 5.07
24.22 2.87
24.39 0.49
25. 25.13
PEROMA
40.38 0.26
39.11
40.30
31.81 0.60
31.71 2.98
33.67
28.78 2.95
28.84 1.06
29.96 1.72
Table 17: Performance comparison across three different non-III Levels of P(X) and three distribu- tion drifting levels the CIFAR-10 dataset.
High
Drifting
5/20
10/20
20
5/20
10/20
20/20
5/220
10
20
FodArg
44.72 14.83
43.09 + 3.40
56.75 9.04
40.86 9.09
45.75 + 9.38
42.29 =
38.35 5.13
37.35 9.07
44.29 = 11.23
FeaRC
43.40 15,66
43.32 4.50
$4.57 8.04
26.99 5.28
43.05 9.04
99411 5.05
27.16 1.97
35.08 9.91
44.42 8.51
13.97
4.74
F-SEM
1
40.48 8.700
19.92 8.95
31,75 10.34
34.09 a 10.76
31.37 9911
7.95
33.42 7.81
34,03 6.70
CRI.
48.02 14:91
44.00 6.16
$5.00 9.65
32.79 5.51
48
30.08 3.12
19.13
44 9.04
41.43 3.54
IFCA
36.96 10.36
41.26
32.50 13.51
27,02 5:27
27.12 9.02
19.76 2
2
18.80
20.10 s
PPedMe
1.766
20.37
20.03
APEL
42.48 9.30
43.23 5.90
39.24 12:88
36,89 6.09
34.81 8.88
25.85 4.95
30,50 4.78
30.89 8.94
29.99 2.388
FedDria
38.37 10.39
41.35 a 8.44
43:44 12.65
33.52 9.66
31.56 9.90
26.85 4.69
27.19 5.98
27.96 4.74
32.92 4,20
a:
29.76
FEROMA
73,04 8.46
73.15 + 3.82
68.69 5.36
58.31 + 5,08
62.88 2..88+t00 6.31
58.60 = 10.85
55.11 3.82
67.47 8.17
58.99 6.57
Table 18: Performance comparison across three different non-III Levels of P(Y) and three distribu- tion drifting levels on the CIFAR-10 dataset.
Non-IID
Medium
Dritting
5/20
Fod/og
4000 + 1.04
39.85 a 1.44
39.65 1.18
36.09 0.97
36.12 L08
35.92 1.07
32.56 1.96
32.15 2:25
32.05 4: 1.79
FolRC
27.06 1.22
26.70 1.53
27.26 1.15
12.17
11.16
12.44
11.31
11.12
10.95 1.00
FolRM
26.73
12.17
12:44 1.76
0.8
FeSEM
40.02 0.65
40.8 0.04
38.36 0.63
37.51 1.02
3
35.78 1.17
36.26 0.63
37.15 0.66
CFL
41.17 1.06
40.90 1:30
4091 0.9%
36.99 1.06
37.09 + 0.83
36.70 1.19
33.39 1.75
$2.93 2.00
32.86 1.92
0..6
38.74
PFeiMe
0.55
0.51
37.19 0.47
1
36.83
a
36.57 0.65
37.27
37.62 0.44
APRL.
42.17
42.59 0.61
40.88
41.66 0.33
41.73
40.44 0.74
40:47
41.32 0.82
FalDDint
0.20
41.04 0.59
0.33
37.09 +
37.13
33.47
34.07
33.28
0.98
1.19
0L32
0.78
41.47-0488
Table 19: Performance comparison across three different non-lID Levels of P(Y and three distribution drifting levels on the CIFAR-10 dataset.
37

---

Published as a conference paper at ICLR 2026
Non- HID Level
Low
Medium
High
Dritting
5/20
10/20
20/ 20
5/220
10/20
20
5/20
10/20
20/20
FodAvg
5.79
23,92
PedRC
22.01 2.43
21.06
25 3.29
18.71
18.01
3.70
16.28
17:35
13.62 1,05
FedEM
20.87 L.85
4.110
22.
4.36
17.29 1.90
18.33 1,56
17.94 1.78
13.71 1.33
16.04 2.11
16.27
PeSEM
28.27 2.44
29.70 4.41
29.65 4.45
25.57 1.75
23.16 3.36
2
23.41 2.41
2
19.97
2:50
IPCA
23.48
29.57 5:28
22.87 + 6.26
19.48 6:33
18.58 + 3,92
16.99 2.15
17.32
11.32.+33.4 3.86
1..54
10:61 0:36
pPodMo
4..0
22.47 4.39
16.79 3:10
16.65 1.25
15.09 2.33
12.75
15.59 1.87
14.09 =: 1.25
13:01
APHL
25.02
2
1.77
24.00
FedDrift
7.017
2
25.30 5.45
0..5
23.20 a
20.53
20.11
3.67
22.37 &
20,99
ATP
27.81 1.81
24.99 6.25
26.08 6.8.1
24.52 1.81
26.54 1,74
24.12 2.24
22.49
1.94
22,87
1.43
21
PEROMA
38.32 3,00
39.89 2.23
38.54
38.12
36.96
35.36 1.86
35,83
31.87 1,76
Table 20: Performance comparison across three different non-III Levels of P(X and three distribution drifting levels the CIFAR-10 dataset.
High
Drifting
5/20
10/
20/20
5/20
/0
20
//20
10//20
20/20
FodAvg
61.86 2.02
63.82 1.54
2.15
69.66 1.31
69:87 + 1.28
0
55.85 1.90
54.46 2.30
55.94 3.49
FaERC:
46.24 36.2400.55 0.55
48.18
50.42 3.99
41.89 3.53
44.1 2.33
46.15
12.55
11:33
0.82
11.65 1.01
12:5 1.70
FeSEM
4.133.4.07
5 3.64
58.77 5.76
65,00
61.55 4.52
66.59 1.45
35.51
38.13 1 7.11
37,48 5.88
CFI.
65.83
66.51 0.99
67.89 1.85
72.15 1.41
71.89
71:92
50:43 1.67
59.64
58.13 1.97
IFCA
27.72 a:
29.86 7.74
33.61 8.65
33,60 a
35.74 8.13
7
19.16
23.13 12.49
10.32
23.66
8:46
23.14
APEL
41.28 1.82
5.37
49.51 2.94
62.39 3.02
64.52 2:27
2..1
38.03 à 6.09
40.97 5,78
41.25 4.07
FedDrif
2:44
33.83 8.95
35.47 3.53
40.77 5.48
45.99 6.78
48.31 2:61
26.69 8.62
28.36 6.97
24.33 4,42
63.19
74.32 a
54.80
FERDMA
73.79 0.41
73,83 a 0.68
73.45 = 0.76
74.84 = 0.49
75.12 0.60
75.03 = 0.51
72.26 = 0.37
71,62 =
71.99 0.65
Table 21: Performance comparison across three different non-IID Levels of P(X) and three distribu- tion drifting levels on the FMNIST dataset.
Non-ILD Level
Low
Medium
High
10/201
29/ 20
20
10/20
20/
FelAvg
77.11
12.18
72:21 1.440
83.51
65.05
68.24 10.91
79.23 5.01
66.09
55.08 16.79
66.47
FedRC
15.70
68.60
75.02
64.14
65.21 9,05
62.11
56.67
53.72 5.36
66.30
FedEM
74.14 9.01
10.95
76.74 8.07
60.32 12.02
64.75 4.05
68.48 12.17
51.0999.0
60.92 10.01
66.79 11.45
FeSEM
73.41
11.73
0..32
69.29 8.1.7
62.55
60.59
67.36 8.50
5035 = 6.67
44.75
64.90 6.96
71.86
IPCA
39.34 12.56
42.07 11.56
37.18 13.17
37.72 15.67
37.57 7.43
7.43
27.73 a
25.37 7.80
21.58 3.61
24.19 6.23
pFedMe
43.05
12.37
44.31 8.35
38.76 17.94
38.08 1 13.36
33.00 11.55
22:77 6.17
$1.35 5.991
24.93
22.75 + 5.67
APFI.
58.41 9.33
56.31
56.
51.51
49.41 9.51
42.26 8.01
FedDuit
45.09 12.05
44.59 11.60
17.64
10.12
34.61
ATP
65.72 19.17
$1.93 14.65
43.36 32.78
61.35 17.68
67.94 18.31
55.95 26.14
48.92 a 27.49
46.53 24.37
46.90 26.28
96.58
97.82
9.55
93.48 4.75
96.09 2.78
97.30 1.88
93.26
95.04 4.29
95.50 2.87
Table 22: Performance comparison across three different non-lID Levels of P(Y) and three distribu- tion drifting levels on the FMNIST dataset.
38

---

Published as a conference paper at ICLR 2026
Non-IID Level
Low
Medium
High
Drifting
5/20
10/ 20
20/20
5/20
10/
20 20
5/20
10/20
20 20/20
FedAvg
5
FedRC
38.95 3:00
58.25
58,66
26.85
30.44
28.24
19.06
FedEM
39.08 3.05
58.23 3.13
58.61 2.71
26.83 11.46
30.43 9.45
28.25 7.45
20.77 4.16
20,40 4.02
19.06 4.59
2.08
66
67.38
64.84
64.04 1.44
63.18 2.53
6347
28.72
55.63 2.12
50.25 3.32
IPCA
69.55 + 3.35
68.59
68.28 4.08
61.89 1.76
60.78 3.52
61.45 2.78
49.74 14.68
50.01 9.74
42.42 16.44
pledMi
76.73 0.34
76.42 0.30
76.30 0.35
76.43 0.68
75.71
0.38
75.91 0.38
75.33 0.24
74.70 0.36
APFL
RelDinn
77.56 0.74
76.93 1.55
72.77 +
75.52 1,28
76.00=0.0 0.60
75.12 0.83
74.75 0.94
73.22 0.50
71. 89 0.97
ATP
64.48 1,70
63 69 2.61
63.63 2.53
56.88 2:56
$6.39 1.96
56.51 3.13
$3.05 #:
$3.50 3.54
$2.00 3.23
FEROMA
75.37 0.33
75.23 0:42
75.25
74.83 2
2.36
74.69
74.29 2
73,83 0.42
73.93 0.26
Table 23: Performance comparison across three different non-lID Levels of PX) and three distribution drifting levels on the FMNIST dataset.
Non-IID Level
Low
Medium
High
Drifting
5/20
10/20
20
5/20
10/
20/20
5/20
10/
20/ 20
FedArg
10.19
09.65
3.94
64.46
66.62
66.23
1.64
61.99 5.29
FedRC
08.93 5,84
5.33
52:11 4.32
43.63 a 4.19
48.53 5.81
34.81 10.03
a
36.21 7.53
FelEM
56.66 e 5.44
59.05
44.11 5.97
43.58
42.91 11.92
44.28
37,80 10.93
8.75
CRI
71.20 1.40
60.80 2.05
72.91 + 2.53
65.72 4.45
66.24 2.24
68.24 1.5..
63.27 + 4.00
66.18 4.57
65.05 2.33
IFCA
34.05 15.83
40.55 # 10.38
25.44 4 11.22
33.76 12.59
30.71 8.15
23:22 0222::6.00
24.94 +
18.30 6.85
23.03 10.05
pPedMe
35.75
30.81
APPL
54,85
56.22 4,92
4
5..80
10.42
4
46.84
FeiDria
40.93 7.29
46.06 5.83
46. 46.16.24 2.88
38.03 10.57
34,23 6.04
2:445
36.15 6.39
37.69 4,47
38.31 2
70.44
69.64
69.1
66.85
66.97
FEROMA
78.74 à 1.78
75.17 5.05
78:96 a 1.49
75.15 a 3.95
76.95 a 3.40
78.12 a 6.37
74.21 a 2.44
76.79 3.44
70.23 7.08
Table 24: Performance comparison across three different non-IID Levels of P(X(Y) and three distribution drifting levels the FMNIST dataset.
Non-IID
Drifting
5/20
10/20
20/
5/200
10/20
20/
5/20
10/20
20
FedAvg
43.85 0.80
4 0.68
44.87 0.87
7.47
13.24 + 4,59
12.84 2:75
10.52 a 8.51
8.58 a 4.26
8.30 3.61
FedRC
44.97 0.73
46.90 0.73
48.07 a: 0.83
13.08 0.64
16.64 4.18
15.70 1966
19.97 5.97
11.93 2.50
13.97 2.67
FiSEM
57.76 1.27
40.33 1.92
43.35 2.04
0..57
18.50 4.07
1 0.40
15.91 3.34
10.63 2.23
12.31 1.79
CFL
43.73
45.90 0.55
46.9% 0.68
13:4 1.82
15.91 + 4.00
14.72 1.56
18.95 5.76
11:94 2.42
11.45
0..19
płedMe
0.85
22:31 0.48
7.17
10.43 0.20
8.23
8/71
APFL
0.25
39.47 1.43
40.66 0.73
12.85 0.59
3.470
1.70
4
12.96 1.77
14.38 2.67
FedDrin
21.36 1.13
23.62
33.93
8.72
13.81 3.54
14.71 1.15
10.16 3.69
10.00 2.13
11.87 1.71
1..77
14.29
3.13 1.04
3,65 0.85
3..83
39.40 0.79
30.37
0.97
Table 25: Performance comparison across three different non-lID Levels of P(X) and three distribu- tion drifting levels on the CIFAR-100 dataset.
39

---

Published as a conference paper at ICLR 2026
Non-HID Level
Low
Mediam
High
Drifting
5/20
10/20
20/
5/.20
10/20
20/200
5/20
10/20
20/
FodAvg
5.48
36,65
40.69 1.77
33.17
37.42
FodRC
34.83
12.36
56.92
42.43
4.58
30.98
42.35
FodiM
50.0 5.49
41.05 6.33
3.44
32.35 5.33
36.00 3.50
4.312
40.32 4.17
4
42.17 3.29
FeSEM
34.43 7.45
35.61 4.14
36.29 3,80
29.09 6.25
32.54
30.65 3:77
31.02 6,26
32.24 1.76
35.33 2.50
3.35
2:77
IFCA
11:28 7.98
10.18
12.97 3.40
2.27 1.28
8.85 8.66
4,66 1.67
2.45
2
2.65 2.49
PFadMe
16.63
13.85
16.55 1.22
11.88 2.94
16.49 +.755
11.26 3.07
13.61 2,90
11.87
11.59 3.26
34,02
31.13 0.77
3.09
2:87
34.72 3.24
FedDrift
24.21 2.2117.611
21.91 4.63
24.89 2.87
18,08 3.98
23.51
17.09
18.05 1 3.02
17.42 2.72
2
AIP
15.05 4.007
36.75 6.32
40.72 4.09
13,88 5.18
36.71 5.550
22490
18.53 10.89
35.31 4.9.1
40.81 4.82
FEROMA
51,03 4.465
51.90
48.43
48.51 2.59
=2.90
40.82
4
45.91
Table 26: Performance comparison across three different non-IID Levels of P(Y) and three distribu- tion drifting levels the CIFAR-100 100 dataset.
Non-IID Level
Low
Medium
High
# Drifting
5/20
10/20
20
5/20
10 20
20
5/20
10/20
20
31.84
38.16
27.39
27.44
27.44 1.20
15.75
15.90 0.40
FedRC
2..91
38.98 0.96
39.23 0.94
28.14 1.12
28.14 1.15
28.16
16.21 2.21
16.19 = 0.40
16.09 0.33
38 0.80
38.95
30.23
28.14 1.12
28.14
28.16
16.19
16.09 0.33
35.99
CFF.
58.28 D.66 0.69
38.61 + 0.94
38.57 0.68
27.78 1.13
1.26
27.63 1.31
16.03 + 0.42
15:92 0.39
15.90 0.41
IPCA
38.17
38.36 0.87
38.54
21.78 1.25
27.62 1.30
27.83
0.97 =
1
0.97 + 0.09
2.191
24.73
24.34 2.2
22.31
APEL
0.19
42,07
42.25 0.48
35,44
0.21
0.28
0.37
29.88 0.24
FedDrift
28. 0.09
26.42 1.00
36.93 1,26
26.56 0.32
25.76 a 0.35
26.87 0.88
23.08 0.32
19:23 a 0.88
15.98 0.60
19.63 1.67
3.30
19.95
1.81
0.79
PEROMA
58.88 0.28
39.62 0.50
59.36 0.30
35.63 0.30
35.71 i 0.29
35.72 a 0.08
31.75 a 0.14
32.46 0.27
32.15 0.25
Table 27: Performance comparison three different non-llI Levels of P(Y|X) and three distribution drifting levels on the CIFAR-100 dataset.
Now-IID JeD Level
Medium
Hlgh
Drifting
FolAvg
33.07 + 7.39
29.13 8.76
28.00 +
23.18 = 7.07
28.56 8.56
21.69 1.68
25.20 8.82
24.30 6.84
27.08 4.887
FaIRC
31.46 7.76
30.58
FaIRM
37.70 6.6
35.00
26.20
FeSEM
27.50
29.57 6:20
3A.66.4..34
22.86
21.38 3:72
25.00 5.06
18.62
3.11
19.88
CFL
28:45 2.88
29.24 6.12
34.74
33.92 4:22
29.60 7.61
25.33 2.770
23.74 4.10
22.52 3.53
24.20
TRCA
13.20
17.00
7.68
PFeiMe
17.02 2.99
11.58
12.74 0.32
10:83 0.79
14.38 4.82
10:37 à: 1.07
11.87
10.72 1.21
1
APEL.
27.51 7.08
29.51 6.17
27.94 3:97
x0.56
29.41 2.92
50.33 3.73
20.25
25.26 2.126-55.16
26.20
FedDDrift
19.13
24.14
19.76
19.21
ATP
1.38
11.15
12.95
7.01 0:733
9.56 1 0.81
12:39
4.55
8.40 2.85
13:70 4.16
1.00
1.31
0.86
Table 28: Performance comparison across three different non-lID Levels of P(X and three distribution drifting levels on the CIFAR-100 dataset.
40

---

Published as a conference paper at ICLR 2026
(subsection I.2); for 20 clients, retain the same dataset size. For larger scales, we increase the dataset size via duplication: duplicating the dataset once for 50 clients, and twice for 100 clients.
Tables 29 Table 33 provide detailed results for FLUX and baseline methods, evaluated across increasing numbers of clients and various distribution shift types and levels. We could not evaluate the performance of FedDrift under 50 and 100 clients due prohibitive memory and computational costs (see Appendix B.2 for details). For the baselines that do not provide a solution for test-only clients, we weight all models by the number of clients in the cluster, and use the expectation that weights all model outputs as an estimation of the predicted labels.
# Clients
Clients
50 Clients
Clients
Algorithm
Accuracy
Time
Accuracy
Time
Accuracy
Time
Accuracy
Time
FedAvg
74.85 a 5.71
3.77
73.98 + 3.04
5,91
74.00
3.33
7.16
73.27 # 3.84
8.28
FedRC
29.84 7.47
8.63
2.35
9.83
25.02
3.94
11.36
24.06 + 3.44
12.38
FedEM
32.01 a 8.56
8.53
26.78
9.27
25.95
4.28
11.27
24.77 +: 3.03
12.11
FeSEM
73.02 + 6.63
7.11
72.58 + 4.65
7.65
73.11
9.12
74.30 3.36
9.93
CFL
77.17
6.32
78.22 4.21
6.57
76.92 4.18
8.52
76,25 + 2,88
8.74
IFCA
37.16 10.44
6.77
45.99 1.99 9.90
8.14
9.66
43.15 5.22
10.66
PFedMe
55.20 12.16
5.65
55.36
6.60
54.40
3,48
7.71
49.77 5.08
9.01
APEL
8.02
5..34
8.38
72.99
10.05
68.39 4.96
10.23
FedDrift
63.34=
57.88 9.440
9.56
N/A
71.39 + 8.95
+ 4.37
6.51
72.46
4.55
7.74
71,21 + 5.45
FEROMA
91.27 0.67
4.84
90.17
1.96
6.43
90.16
1.98
7.67
87.06 2.70
8.61
Table 29: Performance comparison across the number of clients in 10, 20, 50, and 100, summarizing all four types of heterogeneity (P(X), (), P(X on the MNIST dataset. Time reported in log, seconds.
Clients
Clients
20 Clients
Clients
100 Clients
Algorithm
Accuracy
Time
Accuracy
Time
Accuracy
Time
Accuracy
Time
FedAvg
74.20 + 3.72
3.54
74.64 1,50
74.50 + 3.06
7.15
71.05 =
8.16
FedRe
11.44 0.18
8.34
11.35 a 0.00
9.60
11.35 a 0.00
10.95
11.35 0.00
11.43
FedEM
0.14
8.07
11.35 + 0.00
8.80
11.35 + 0.00
10.13
1.00
11.49
FeSEM
3.55
6.90
72.98 4.49
7,30
74.54 1.80
8.50
1.34
9.38
CFL.
79.66
6.47
81.08 + 2.36
6.38
78.73 + 3.18
8.40
77.04 0
8.52
IFCA
22.70
6.51
39.79 4.79
7.56
47.86 + 9.25
9.43
42.23
10.15
pFedMe
52,90
5.57
46.24
6.37
6.66
50.62
3.74
7.63
47.22
1.47
9.06
APEL
72.89
7.01
71.85
7.94
74.07
0.96 N/A
9.97
72.40
1.04
10.12
FedDrift
5.80
8.74
46.86
8.80
9.55
N/A
NIA
N/A
7.72
3.95
0.56
7.79
1.54
FEROMA
87.51 + 1.19
4.00
86.45 + 1.22
6.32
87.69 + 0.91
7.88
83.09 1.29
8.87
Table 30: Performance comparison across heterogeneity type P(X) on the MNIST dataset. Time is reported in log, seconds.
I.4 RESULTS ON WORLD DATASETS
Following the setup in subsection I.I, we compare the performance of FEROMA against baseline methods on both the CheXpert and Office-Home datasets (baseline ATP cannot be adapted to the multi-label classification task of CheXpert dataset). As shown in Table 34 and Table 35, FEROMA consistently achieves top-tier performance across various settings of non-llID severity and degrees of distribution drift. Specifically, either outperforms or closely matches the best besf-performing methods in nearly all configurations. While personalized methods such ATP and APFL occasionally yield strong results- particularly in low-drift scenarios, their performance tends to degrade under increasing drift levels, indicating overfitting local data. In contrast, FEROMA maintains stable and high performance even in highly drifted or non-IID environments, demonstrating strong robustness and generalization without relying on explicit personalization.
41

---

Published as a conference paper at ICLR 2026
Clients
10 Clients
Clients
Clients
100 Clients
Algorithm
Accuracy
Time
Accuracy
Time
Accuracy
Time
Accuracy
Time
FodAvg
85,98 6,45
4.34
84.82 + 4,58
6.19
80.64
3.01
7.30
84.38
6,68
8.41
FodRC
68.89 10.89
8.88
75.08 + 5.67
10.32
63.79
5.66
11:52
60.82
6.26
12.25
FedEM
72.31 9.84
8.85
66.08
5.81
9.42
67.11
5,95
11.19
4.71
11.87
FeSEM
78.62 6.58
7.44
72.37 3.53
7.97
75.11
9.19
77.53
9.88
CHL
9.05
IFCA
36.77
7.10
10.01
37.22
10.74
PFodMe
38.59 18.23
6.06
39.32 13.26
7.00
34.31 4.61
25. 25.16 7.59
9.53
APFL
$3.90 13.88
7.68
56.56 + 8.56
8.70
55.12 6.09
10.60
47.32 8.51
10.60
FedDrint
45.60 16.15
8.87
43.46 # 14.96
9.69
N/A
N/A
N/A
N/A
AIP
80.42 10.16
4.93
86.25 + 7.72
6.62
81.29 + 8.67
7.91
74.85 + 8.51
8.79
Table 31: Performance comparison across heterogeneity type P(Y) on the MNIST dataset. Time is reported in log, seconds.
Clients
10 Clients
20 Clients
Clients
Clients
Algorithm
Accuracy
Time
Accuracy
Time
Accuracy
Time
Accuracy
Time
FedAvg
66.16 1.87
3.45
65.75
1.63
5.88
63.30 a: 0.91
7.20
62.92 a 0.69
8.37
FeaRC
13.84 4 3.24
8.98
12,91
2.29
9.87
10.50 a: 0.42
11.90
11.48 + 1.49
13:45
FodEM
13.84 +324
8.85
12.91
2.29
9.81
10,50 + 0.42
12.22
11.49 # 1.49
13.11
FeSEM
2.117
7.17
75.43 + 1.55
7.90
73.89 # 1.31
9.84
70.97 + 0.67
10.67
CFI.
68.73
5.35
67.26
1.23
6.25
65.71 + 0.75
8.14
64.87 + 0.83
8.72
IFCA
69.66 1,58
6.90
69.70
4.02
8.84
70.14 4.59
9.95
69.12 5,36
11.32
piedMe
0.32
5,32
90,95
0.47
6.17
90.01 0.36
7.38
88.39 0.46
8.32
APEL
91.85 $1.85
7.64
91.61 0.68
8.36
91.05 + 0.35
90.09 0.62
9.99
FedDrift
95.56
8.53
96,59 0.26
9.41
NIA
N/A
63.24
1.53
8.56
FEROMA
88.81 0.40
5.74
88.59 0.62
6.24
88.40 0.47
7.44
88.10 1 0.57
8.44
Table 32: Performance comparison across heterogeneity type X(Y on the MNIST dataset. Time reported in log2 seconds.
Clients
10 Clients
20 Clients
50 Clients
100 Clients
Algorithm
Accuracy
Time
Accuracy
Time
Accuracy
Time
Accuracy
Time
FedAvg
73.04 8.44
3.57
70.71
3.32
5,76
77.57
5.02
6.98
74.72
1.54
8.13
FedRC
12.57
FedEM
30.45
16.79
14.10 3.52
11.10
FeSEM
62.78 10.71
6.84
+ 7.16
7.26
68.90 1.84
8.53
73.22 3.80
9.39
CFL
78.74 4.27
6.16
79.31 + 5.43
6.49
81.35 + 4.08
8.46
80.69
1.53
8.64
IFCA
8.37
6.47
39.20
15.89
7.55
38.37 + 5.22
8.98
34.44 + 4.84
10.04
PFodMe
38.00
5.57
44.92 2 9.61
6.44
42.67 + 3.61
7.64
38.31 1 6.57
8.89
APFL
67.09
7.16
68.95 4 4.98
8.41
71.71 + 5.63
9.94
63.73 4 4.93
10.12
FedDrift
52.08 + 15.30
8.79
44.62 + 7.23
9.58
N/A
N/A
N/A
N/A
ATP
67.67 12.49
5.17
72.62
1.79
6.67
68.87 = 2.28
7.73
73.47
6.46
8.53
FEROMA
89.64 = 0.29
4.05
80, 19.9004 + 0.63
6.60
89.77 = 0.82
7.60
88.04 + 1.41
8.37
Table 33: Performance comparison across heterogeneity type P(X on the MNIST dataset. Time is reported in log, seconds.
42

---

Published as a conference paper at ICLR 2026
Non-HID Level
Mediam
Dritting
10/20
5/
5/20
FodAvg
59.90 a: 1.04
62.07 a
57.68
2.05
60.91 a 1.19
59.34 5.35
61.26 a 1:25
57.38 1.17
$6.76 2.00
56.62 a 1.31
57.07
56.77
55.79
52.76
PedLi
56.31 1.27
024
3.42
53.58
53.45
52.44
30.45:2..8
2.334
50.44 2.52
FoSEM
64.09 3.63
6
0.118
19.33 0:29
00.76 0.18
61.32
59:23 2.45
58.55 =
57.54
CFL
63.09 5.221
64.85 0.07
63.73 0.224
62.34
62.21 1 0.22
60:44 6.13
61.43 a 0.83
60.43
1.54
PRadMe
0.67
61.12 2.78
58.68 1.46
0.332
0.11
57.88
55.14 0.04
55.48 0.03
56,07 0.25
APFI.
1.23
56.90 0.06
$5.90 + 0.18
$4.78 1.72
0.53
53.55 1:35
53.78 0.28
$2.83 0.12
51 + 1.02
Pedibrift
74.86
72.02
N/A
N/A
N/A
NM
N/A
N/A
N/A
N/A
71.96
71.82
Table 34: Performance comparison across three different non-III Levels and three distribution drifting levels on the CheXpert dataset.
# Dritting
5/20
10/20
20
FedAvg
41,83
40.11 1.42
41,02
FuiRC
a.30
13:52 1.44
FedEM
14.3244..0
14.96 2 0.50
4.54
CHL
36.27
33.93 = 0.77
34,33
IPCA
40.00 4.54
37.17 2:12
22.83 5.90
phodMe
APPL
2.99
39.99 0.30
24.08 2.20
FalDrih
42.08
42.34
41.94 1.12
5.25
TEROMA
4..43
42.75 a 1.21
41.11 1.95
Table 35: Performance comparison across three distribution drifting levels the Office-Home dataset.
I.5 COMPARISON WITH NON-FL TEST-TIME ADAPTATION APPROACH
In this section, we compare FEROMA with CoLA (Chen et al., 2024a), which performs collaborative knowledge vectors across devices and updating each test-time adaptation by sharing compact reprogramming-bee
adjustments handle domain shifts during inference. Despite that CoLA is not originally proposed for FL., we include it as a strong baseline as its design aligns with FEROMA'S ability handle test-time distribution changes.
client using lightweight similarity- or
Non-IID Level
Low
Medium
High
# Drifting
5/20
10/20 20
20/200
5/20
10 20
20/200
5/20
10//00
20/20
CoLA
5..0
2.3
82.2 8.1
843
8..
84.6
FedAvg
66.5 a 1.2
67.9 1.5
72.1
4.0
73.7 2.3
73.2 1.8
54.8 188
54.3 2..3
5.95
PEROMA
0.3
88.7 0.3
89.5
87.1
2.3
86.5 2.5
86.1 3.4
864
2..8
85.7 1.2
Table 36: Performance comparison among CoLA, FedAvg, and FEROMA with heterogeneity type P(X), across three different non-IID Levels and three distribution drifting levels.
The experiment settings are the same as reported in our main experiments. As shown from Table 36 to Table 39, CoLA provides strong test-time robustness due to its domain-adaptation mechanism, but it does not address training-time distribution drift; consequently, while its performance competitive, it does not surpass FEROMA, which handles both drift and shift during training.
ehheeimeeen
I.6 GENERALIZABILITY OF DPE TO TIME- TIME-SERIES DATA
FEROMA does not make any assumption about the underlying data modality. Instead, it relies on the DPE as an abstraction layer that compresses potentially high-dimensional, multi-modal, or long-tailed client data into compact distributional representation. As long a suitable feature extractor exists
43

---

Published as a conference paper at ICLR 2026
Non- Level
Low
Medium
High
Drifting
20/
10/20
CoLA
=4.7
98.7 0.5
99.2 1.5
94.2
99.0
94.0=
90.6 7.6
96.7 a 244
FodAug
90.8
74.7
PEROMA
96.5
2..7
97
98.7
9 1.9
97.2
Table 37: Performance comparison among CoLA, FedAvg, and FEROMA with heterogeneity type P(Y), ), across three different non-llI -ID Levels and three distribution drifting levels.
Non-HID Level
Low
Medium
High
* Dritting
5120
10/ 20
20/ 20
5/20
10 20
20/220
5/
10//0
20/23 20
CoLA
2.
85.4 1.5
76.5
7..1+
= 188
0.4
77.8
FadAvg
73.5
72.9 0.8
72. 1.0
64.9 1.7
65.0 2.1
64.2 2.0
-7.1
57.2 2.0
56.4
FERDMA
09.6
89.5 0.3
89.8 0.
88.9 0.5
88.9 0.7
89.0 0.5
882 007
88.1 0.5
88.4 0.7
Table 38: Performance comparison among CoLA, FedAvg, and FEROMA with heterogeneity type P(Y [)), across three different non-IID Levels and three distribution drifting levels.
for a given modality, FEROMA can operate on top of it without modification to the aggregation or mapping mechanisms.
In the main experiments, we instantiate the DPE using few-round trained models for image- based tasks, this provides practical and lightweight way capture distributional structure while remaining easy to integrate into standard FL pipelines. Importantly, this choice is not intrinsic to FEROMA. The DPE design is intentionally model- and can be instantiated with alternative feature extractors tailored to different data types.
To evaluate the generalizability of the DPE abstraction beyond image data, we adapt FEROMA to time-series classification task using the UCI Human Activity Recognition (UCI-HAR) (Reyes-Ortiz al., 2013) dataset. This dataset consists of multi-dimensional sensor signals and exhibits strong temporal dependencies and multi-mode feature characteristics. Due to its relatively small scale, we set the number of clients structure. Client heterogeneity is introduced via Dirichlet sampling over label distributions with concentration parameter a, where smaller a corresponds to stronger non-IIDness in P(Y).
and avoid data augmentation order to preserve its real-world
Table 40 reports the results across different heterogeneity levels. FEROMA consistently outperforms all baselines by large margin under severe heterogeneity, and remains competitive as heterogeneity decreases. These results indicate that coabbutioo--rffile-bbb aggregation remains effective even when client feature spaces are dime-dependent and multi-mon motti,, and that the DPE abstraction is not limited to image-based representations.
1.7 FEROMA ROBUSTNESS AGAINST UNSEEN OR OUT-OF-SUPPORT DISTRIBUTIONS
FEROMA is not explicitly designed to handle completely unseen or out-of-support data distributions. Nevertheless, robustness to such scenarios is implicitly evaluated throughout our experimental setup. In the drifting dataset generation process, test-time distributions may differ from those observed during training, particularly under high non-IlI severity and low drifting frequency (e.g., non-III
Non-HID Level
Medium
Deifting
CoLA
$6.7 0.2
89.0 1.7
3..
82.6 7.2
0.
88.1 0.
79.2 6.6
88.9 0.6
FalAvg
73.3
73.1
7.00+41
716-5.1
69.9
FIROMA
1.9
87.0
89.0
90.12
894 1.2
89.7
Table 39; Performance comparison among CoLA, FedAvg, and FEROMA with heterogeneity type P(X Y),, across three different non-III Levels and three distribution drifting levels.
44

---

Published as a conference paper at ICLR 2026
PodAvg
FEROMA
FedRC
FedEM
PeSEM
IFCA
APEL
FedDrift
ATP
61,5+33
309
39.9+49
6,4
38.5+78
34.0
2,5+9.1
2.
53
4.
37.9 29.8
4.
0.6
3.0=50
58.0
40.5 442
40.5+43
7.
9..0
30.7
46229.
3.7
41.3+8.3
0.7
37.7
5.99179
4..1
41.2445
28.1
44.627.1
5..6
309 309 13.3
46.6.55
38.3 10.5
43.9
43.
2.5
42.
43.0.88
6.7
41.5
43.
Table 40: Performance on the UCI-HAR time-series dataset under varying label distribution hetero- geneity. Smaller indicates stronger non-IIDness in P(Y).
level High with drifting 5/20 in Table 2). These cases, analyzed in Appendix B, naturally include test distributions that are not present in recent training rounds.
To more explicitly evaluate robustness against truly unseen distributions, conduct an additional experiment on MNIST with 20 clients under medium non-llI conditions. During testing, we vary the proportion of clients whose data distributions are unseen during training, and report the final test accuracy averaged over drifting frequencies 5, 10, and 20 (out of 20 total rounds).
Table 41 reports the results. When clients encounter unseen distributions, FEROMA achieves its standard baseline performance. As the unseen rate increases, FedAvg exhibits steady degradation in accuracy. In contrast, FEROMA maintains strong performance even extreme settings where all clients face unseen test distributions, indicating that profile-base model assignment and aggregation generalize beyond the training support.
Unseen
FedAvs
73.25
4.45
69.12 2.84
65.20 a 0,76
62,60 0.31
59.59 a 0,78
53.05 a 4.17
86 1.56
85.79
85,
0.00
Table 41: Robustness to unseen test distributions on MNIST with 20 clients under medium non-IID conditions. Results are averaged over drifting frequencies 5, 10, and (out 20 rounds).
Overall, these results indicate that although FEROMA does not explicitly model out-of -ut-oo-support distributions, its Distribution-profiie profile- based aggregation and model assignment mechanisms provide strong empirical robustness under severe test-time distribution shifts.
1.8 ABLATION STUDY ON MONTE MONTE-CARLO SUBSAMPLING
The Distribution Profile Extractor (DPE) must satisfy Requirement (R3) on controlled stochastic- ity. As described in Section B. 1. our implementation enforces this property via a Monte Carlo subsampling strategy. Specifically, for each client, we apply M independent Bernoulli masks with sampling probability 7 0.5. and compute distribution profiles over the resulting subsets. The variance introduced by this process is governed by the number subsamples M, the number used data points (**), and the proxy variance 22 of the latent coordinates, as derived in Equation Among these parameters, MM and are design choices under our control. In this ablation, we study the impact of varying M E 2,3. keeping 7 = 0.5 fixed, across four types of distribution shifts (P(X), P(Y), X(Y]X), P(X(Y) and three heterogeneity levels (low, medium, high), on both MNIST and CIFAR-10.
The results, reported in Tables 42-45, show that FEROMA is robust to the choice of M, with only minor accuracy differences across values. Nevertheless, slightly better performance observed with larger M, suggesting that reducing the variance of the extracted profiles improves model association and final accuracy. This supports the intuition that while stochasticity is essential for privacy and regularization, excessive noise may degrade the reliability of distribution similarity computations.
1.9 ABLATION STUDY ON THRESHOLD PROFILE ASSOCIATION
To prevent noisy weakly related distributions from influencing model aggregation, we apply thresholding mechanism to the profile similarity weights. Specifically, a threshold r introduced to discard low-similarity associations, thereby promoting aggregation only among clients with suffi-
45

---

Published as a conference paper at ICLR 2026
ciently aligned distributions. This design is motivated by clustered FL principles, where collaboration is restricted to similar clients, but implemented here in a soft and data-driven manner. The threshold-
ing step is applied after computing the similarity-based weights w; (k.)), as described in Equation 4. In the ablation study, we set the threshold T to the average value the similarity weights (e.g.. 0.05 in the case of 20 clients), providing simple heuristic to evaluate the effect of thresholding.
From Table 42 to Table 45, the results show that accuracy remains largely stable with thresholding enabled disabled. In future work, we expect that more principled or adaptive strategies for selecting the threshold T could further enhance aggregation quality quality-particllan particularly in highly imbalanced or noisy distribution settings.
Non-IID Type
P(X)
P(Y)
Non-IID Level
Low
Medium
High
Low
Medium
High
M I. threshold 87.88 = 1.07
87.29 = 1.48
85.30 1.34
84.56 9.61
79.69 10.61
79.85 10.41
# M 2, threshold off
88. 0.88
87.55 a 1.61
85.89 0.30
87.61 = 6.53
83.81 8.62
78.91 # 10.86
# M
3, threshold off
88.67 0.30
06.50
85.81 a 0.36
86.24 =
85.94 2
80.11 + 9.65
# M
1. threshold on
87.62 a
1:1111.44 87.21
85.71 a 0.52
86.05 2 8.60
76.37 10,30
13.26 4: 11.61
# M
2, threshold on
87.53 + 1.00
85.61 4.75
85.48 + 0.46
86.95 + 7.53
81,59 8.67
75.75 + 7.90
# M 3, threshold
88. + 0.42
85.98 + 2.10
85.43
0.27
86, 80 9.44
79.89 8.26
68.67 12.67
Table 42: Test accuracy on MNIST under varying numbers of Monte Carlo masks (M = 1, 3) and with thresholding enabled or disabled, across different non-IID levels in P(X) and P(Y).
Non-IID Type
P(Y
P(X
Non-IID Level
Low
Medium
Hlgh
Low
Medium
High
# M 1. threshold off
84.10 2.90
76.40 + 3:70
73.88 + 1.75
90.02 = 0.47
88.64 + 1.21
89.67 : 1.14
# M 2, threshold aff
84.40 + 1.36
78.71 + 3.78
77.12 3.07
88,70 2.12
90.06 + 0.23
88.53 + 0.96
# M
3, threshold off
85.04 + 2.44
80.09 2.14
78.38 + 2.62
88.81 + 1.94
89.02 + 1.59
88.29 1.12
# . 1. threshold 84.09 2.87
76.54 a 3,68
73.76 1.75
89.38 0.87
88.24 2.52
88.88 0.93
# M 2, threshold
84.50 3.25
78.59 3.68
76.99
3.10
89.
0.41
88.26 2.04
86.03 3.24
3, thresbold
85.02 2.43
79.90 2.03
78.05
2.57
90.05 0.69
89.47 1.35
88.86 1,23
Table 43: Test accuracy on MNIST under varying numbers of Monte Carlo masks (M = 1, 3) and with thresholding enabled or disabled, across different non-Ill) levels in P(Y and P(X[Y).
Non-IID Type
P(X)
P(Y)
NIn-IID
Medium
High
Medium
High
# M 1. tbresbold off
40.25 a 0.79
32.76 a 2:43
30.97 1.05
39.54 1 5.29
34.73 1 5.06
31.16
3,31
# M 2, threshold off
40.1 0.57
32.41 2.78
31,05 1.73
38.73 6.90
35.62 5.37
32.24 9.06
3, threshold off
40.13 0.73
32.94 1.78
31.61 1.69
41.01 +
41.33
33.94 7.60
threshold
2.44
5.73 39.01
38.91 1.38
30.24 3.50
2. threshold
32.23
8.30
# M 3, threshold on
40.19 2 0.60
32.10 1.57
30.76
1.28
39.98 + 8.64
41.48 + 3.04 31.64 5.89
Table 44: Test accuracy CIFAR-10 under varying numbers of Monte Carlo masks (M = 1,2,33 and with thresholding enabled or disabled, across different non-IID levels in P(X) and P(Y).
I.10 ABLATION STUDY ON DISTANCE FUNCTION
We analyze the impact of the distance function D(..), introduced in Equation 3, on the overall performance of FEROMA. Specifically, compare commonly used similarity measures: Euclidean distance and cosine distance. To isolate their effects, we evaluate four combinations by varying the choice of D during training and test-time profile matching.
The evaluated settings are as follows:
E.E.: Fuclidean distance used in both training and testing.
46

---

Published as a conference paper at ICLR 2026
Non-HID Type
P(YIX)
P(X)))
Non-IID Level
Low
Medium
High
Low
Medium
High
# . 1. threshold off
36.83 + 1.91
34.16 1.30
29.47 + 0.50
41.56 + 1.72
39.77 + 1,30
38.94 + 2.85
# M 2. threshold off
36.24 0.36
34.20 1.47
30.01 + 0.99
43.91 + 1.36
37.93 1.41
37.59 + 3.65
# M
3, threshold off
36.78 + 1.74
34.48 + 1.85
29.91 + 1.00
3.775
42.38 + 1.82.
38.91 1.89
# . 1. threshold 36.83 1.53
34,23 1:22
29.64 0.55
40.22 3.331
40.30 a 0.99
37.93 1.56
# . 2. threshold
36.32 a 0.67
34.49
30,07 1.22
41.62 a 3.37
38.71 2:85
34.21 2.71
3, threshold
36.94 1.42
34.55 1.72
29.94 0.93
38.17 2.85
39.04 2.68
37.25 1.51
Table 45: Test accuracy on CIFAR-I0 under varying numbers of Monte Carlo masks (M = =1, and with thresholding enabled or disabled, across different non-IID levels in P(Y and P(X
C.C.: Cosine distance used in both training and testing.
E.C.: Euclidean distance used during training; cosine distance during testing.
C.E.: Cosine distance used during training; Fuclidean distance during testing.
These combinations allow us to assess whether consistency between training and testing distance functions is important, and whether certain metrics generalize better under mismatch. The ablation study is conducted on the FMNIST dataset using all four non-IID types described in the main experiments. For each type, we evaluate three non-IID levels (Low, Medium, High) and three distribution drift levels (5/ (5/20, 10/20, 20/20), following the same settings as detailed in subsection I.2. Table 46 summarizes the overall performance across all combinations, and the result shows FEROMA is robust despite the choices of 'D. We adopt the C.E. configuration (cosine distance for training and Euclidean distance for testing) for the remainder of our experiments, consistently achieves slightly better performance. Detailed results for each non-lID type provided in Table 47 through Table 50.
Drifting
5/20
10 20
20 20
E.C.
75.911 1:001:=:.11 3.12
76.499 + 3.94
75.930 5.75
E.E. 77.833 3.43
77.079 + 3.76
76.603 + 4.59
C.E.
77.904 + 3.42
77.357 + 3.93
76.891 4.73
C.C.
75.864 3.01
76.302 3.78
75.809 5.75
Table 46: Performance comparison among different distance functions across all non-lID types and levels the FMNIST dataset.
High
Drifting
5/20
10/20
20/
5/20
10
20/20
5/20
10/20
20 20
E.C.
73.73
74.89 a 0.43
2::13:0::43
73.76 0.73
75.11 1 0.67
71.58:00.5
73.43 0.75
75.09 a 0.48
72.08 a 0.62
E.
73.63 0.53
7487 0.57
71.99 0.25
75,09 0.70
75.01 0.63
71.30
73.65 + 0.69
75.01 = 0.50
7
75.04
C.C.
73.79 0.41
74.84 0.49
72.26
73,83 0.68
7
=1362:00.4
73,45 0.76
75.03 e 0.51
71.99 0.65
Table 47: Performance comparison among different distance functions across non-III type P(X) on the FMNIST dataset.
Non-IID Level
Mediam
Deifting
5/20
10/20
10/
E.C.
94.11 2.60
$2.89 7.27
83.09 4.44
95.05 3.45
90.20 6.38
84.29 8.43
94.07 6.25
90:46 8.166... 6.10
85.09 14.84
14.49=2.70
6.23
94.44
90.2
78.27 9.74
93.99
90.00
13.77
2..66::2.65
38.38 6.34
W6.22
9.08
9
90.33 6.92
81.01
90.05
3.73
13.43
C.C.
93.82 =
82.93 7.32
$2.98 3.56
05.01 3.48
6.277
$3.51 = 7.60
93.88 6.57
90.34
83.97 14.66
Table 48: Performance comparison among different distance functions across non-IlI type P(Y) on the FMNIST dataset.
47

---

Published as a conference paper at ICLR 2026
Non-IID Level
Mediam
Drifting
5/220
E.C.
72.52 1.69
66.83 2,61
65.28 1.21
11.00 2.71
69.14 1.41
63.29 1:54
72.41 1.63
67.01 a 2.00
63.03 1.81
0.57
72.26
72.10
70:21
73.68 0.80
7..44
71.83
72.38
70.3
06.87 2.14
64,00 1.22
C.C.
72.49 1.69
6 1.95
66.11
70,62
2.72
68.61 1.40
63,47 1.57
72.48
1.64
66.74 1.66
63.08 2.01
Table 49: Performance comparison among different distance functions across non-IID type P(Y on the FMNIST dataset.
Now IID Level
Low
Medium
Hlggh
5/20
E.C.
78.55 1.92
75.55 396
74.62 2,53
75.14 5.5
17.27 + 3.61
76.74
79.02 1.49
73.06 6.16
70.34 7.06
79.22
79.33
79:44
C.E.
78.88
1.68
79.11
1.99
0.88
78.81 1.39
78.42
78.14
79.55
79.75 & 1.83
78.85 0.94
C.C.
78.74
1.78
75.15 3.5
74.21 2.44
75.17
76.95
76.79 3.44
78.98 1.49
63.12
70.23
Table 50: Performance comparison among different distance functions across non-IID type P(X on the FMNIST dataset.
J ILLUSTRATIONS THE DYNAMIC AGGREGATION STRATEGY
This appendix visualizes the dynamic aggregation behavior of FEROMA using 20 heatmaps, each corresponding to a single client. Each heatmap shows the evolution of profile distances to other clients across training rounds, illustrating how aggregation weights adapt time. Dynamic aggregation begins at round 6; in this round, all distances are initialized to a uniform value (0.05 for 20 clients) due to the absence of previous-round profiles for comparison.
Client Distance Matrix for Client
Client Distance Matrix for Client
0.5
0.4
04
Round Number
10
Divanos Vilu
10
12-
13
13
14
14
15
15
16
16
17
17
0.1
19
20-
0.0
20
0.0
Provious Round Clint Index
Drevious Round Client Index.
48

---

Published as a conference paper at ICLR 2026
Client Distance Matrix for Client 3
Client Distance Matrix for Client 4
0.4
Bound Beund Number
Diitunss Vilte
0,3
12
12
14
13
0.3
14
13
18
18
10-
19-
20-
20-
0.0
10 15
11
15 16 19
Provious Round Climnt Index
Previous Round Cliem Indee
Client Distance Matrix for Client
Client Distance Matrix for Client
0.6
10-
Round Number
12
12
13-
14-
0.2
14
13
0.2
17-
18-
18
0.1
19-
19
20
20-
1011 11
19.20
10.20
Previous Round
Drevious Round Cliem Index
Clicnt Distance Matrix for Clicat 7
Client Distance Matrix for Client
Disanoo Vilue
0.4
12-
0,3
12
13
13
0.3
14
14
0.2
18-
18
-01
19
19
20-
20
12
14
10
15 16
19.20
Previous Round Client Index
Previous Round Client Indox
Client Distance Matrix for Client 9
Client Distance Matrix for Client 10
0.6
06
0.5
0.4
12
12
13-
14
0.3
Diilanc
14
13
0.3
0,2
18
0.1
10-
19
20-
0.0
12
14
14
Provina Round Client Indes
Previous Round Cliem Index
49

---

Published as a conference paper at ICLR 2026
Client Distance Matrix for Client 11
Client Distance Matrix for Client 12
0.5
0.5
0.4
Bound Beund Number
Diitunc Valae
12
0.4
12
0,3
13
13
14
0.3
14
0,2
18
18
0.1
10-
19-
20-
20-
0.0
12 15
10
14
19
Provious Round Climn Index
Previous Round Cliem Indee
Clieni Distance Maltix for Clieni 13
Client Distance Matrix for Client 14
0.6
0.5
Samber
Di Distanc Valee
12
13
13
14
0.
14
0.2
18
18
0.1
10
19
20-
20
0.0
12
14 16
14
Provious Round Clicat Index
Previous Round Cliem Index
Client Distance Matrix for Client 15
Client Distance Matrix for Client 16
0.5
0.4
10
00A
Dis iistenee Valae
Bound
12
12
11
1
14
14
15
0.2
18
18
0.1
19-
19
20-
0.0
20
0.0
10
14
10
14
19
Previnus Roand Client Index
Previaus Round Client Index
Client Distance Maltix for Client 17
Cleent Distance Matrix for Client 18
0.6
10
0.5
Ntaber
Pilttnc Valte
13
13
14
14
0.1
0.2
18
10-
19
20
20
0.0
11
14 15
18
10
14
Previna Round Client Indes
Previous Round Cliem Index
50

---

Published as a conference paper at ICLR 2026
Client Distance Matrix for Clicnt 19
Client Distance Matrix for Client 20
0.5
Round Number
10
0.4
Distuoo Vilue
10
-04
Disunor Valoe
0,3
13
13
14
15
15
0,2
16
10
0.1
19
19
20-
0.0
20
0.0
Provious Bound Clinat Index
Previous Round Client Indox
K EVALUATION WITH HIGHER NUMBER OF COMMUNICATION ROUNDS
This appendix reports additional experiments CIFAR- 100 with 20 clients under medium non-IlI) heterogeneity of type P(X), using 100 and 200 communication rounds with drift introduced every 10 rounds. We plot per-client test accuracy over time to examine long-horizon training behavior. While FedAvg exhibits large performance fluctuations and unstable convergence at each drift event, FEROMA maintains steady accuracy improvements with only minor transient variations, demonstrating stable adaptation under prolonged dynamic non-IID conditions.
Accuracy per Round for All Clients (100 Rounds)
0.8
FedAng Clients
0.6
Accuracy
0.2
0.0
20
40
60
80
100
Number of Rounds
51

---

Published as a conference paper at ICLR 2026
Accuracy per Round for All Clients (200 Rounds)
1.0
FEROMA Clients
FedArg Clieats
0.8
0.6
Accuracy
0.4
0.0
25
50
75
100
125
150
175
200
Number Rounds
L LARGE LANGUAGE MODEL USAGE DISCLOSURE
We disclose the use of Large Language Models (LLMs) in the preparation of this manuscript. Specifically, we used Claude (Anthropic) and GPT-4o solely for writing assistance and polishing. LLMs were used exclusively for:
1. grammar correction and sentence structure improvement,
2. clarity enhancement and readability optimization,
3. consistency in technical terminology and notation, and
4. general writing style refinement.
No content, ideas, analyses, experimental results were generated by LLMs. All suggestions were carefully reviewed, edited, and approved by the authors before incorporation. The authors retain full responsibility for the entire content, including any errors or inaccuracies.
52


Reviews:
[{"review_id": "0jfdg6svZY", "reviewer": "Reviewer_aPnB", "summary": "This paper proposes FEROMA, a FL framework designed to tackle both distribution shifts across clients and distribution drifts within clients over time.  FEROMA introduces privacy-preserving distribution profiles, a lightweight representations that capture each client's data distribution. Through these profiles, FEROMA adaptively maps client data to previous distributions, guiding both aggregation during training and dynamic, test-time model selection for unseen or unlabeled clients. Experiments across various benchmarks and real-world datasets show improvements.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. FEROMA introduces a differentially private distribution profile for each client.\n2. The method address diverse FL scenarios in a single framework.", "weaknesses": "1. This paper lacks some necessary to compare and discuss in the related work and experiments, e.g., [R1-R3]. Recent alternative approaches for handling distribution shifts and drifts in FL are omitted [R4-R5].\n2. How would the DPE behave under significant violations of its underlying assumptions—e.g., highly multi-modal, long-tailed, or high-dimensional client feature spaces? Is there any empirical or theoretical work supporting extension of the current implementation to natural language or time-series data?\n3. How robust is FEROMA to adversarial profile manipulation or dishonest client participation? If malicious clients attempt to game profile similarity, what is the effect on model assignment and privacy?\n4. The DP mechanism claims negligible performance reduction at $\\varepsilon=10$, but Table 9 shows nontrivial drops for stricter budgets. The risk model and privacy guarantees need a more critical security-oriented discussion. \n\nReferences:\n\n[R1] Tan Y, Long G, Liu L, et al. Fedproto: Federated prototype learning across heterogeneous clients[C]//Proceedings of the AAAI conference on artificial intelligence. 2022, 36(8): 8432-8440.\n\n[R2]Yang Z, Zhang Y, Li C, et al. FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning[J]. arXiv preprint arXiv:2510.20250, 2025.\n\n[R3]Chen G, Niu S, Chen D, et al. Cross-device collaborative test-time adaptation[J]. Advances in Neural Information Processing Systems, 2024, 37: 122917-122951.\n\n[R4] Liao X, Liu W, Zhou P, et al. Foogd: Federated collaboration for both out-of-distribution generalization and detection[J]. Advances in Neural Information Processing Systems, 2024, 37: 132908-132945.\n\n[R5] Bhope R A, Jayaram K R, Venkateswaran P, et al. Shift Happens: Mixture of Experts based Continual Adaptation in Federated Learning[J]. arXiv preprint arXiv:2506.18789, 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WPZtVBNuZh", "reviewer": "Reviewer_pNbg", "summary": "This paper introduces FEROMA, a new Federated Learning (FL) framework designed to handle both distribution shift (heterogeneity across clients) and distribution drift (temporal heterogeneity within clients). Instead of relying on client or cluster identities, FEROMA constructs distribution profiles, including compact and differentially private summaries of local data, to guide model aggregation and test-time model assignment. FEROMA adaptively transitions between clustered, personalized, and global aggregation based on profile similarity, allowing it to generalize to unseen clients without retraining. Extensive experiments across six benchmarks (MNIST, FMNIST, CIFAR-10/100, CheXpert, and Office-Home) show up to 12% accuracy gain over 10 baselines with comparable overhead to FedAvg.", "questions": "- The paper claims computational and communication costs comparable to FedAvg. Could the authors quantify this in terms of total transmitted bytes or runtime for larger models (e.g., ResNet or Transformer architectures)?\n- The paper includes many experimental tables and dense numerical results. Could the authors consider summarizing key findings in more digestible form? A short “Key Observations” paragraph at Section 4 might help readers connect results back to FEROMA’s core design principles.\n- Does FEROMA automatically adapt this threshold during training, or is it fixed?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ This paper contains extensive experimental evaluation. The paper includes thorough comparisons with 10 strong baselines across six datasets (including real-world ones like CheXpert and Office-Home). Performance gains are consistent (e.g., +14.1pp on MNIST, +10.3pp on CIFAR-10, Table 1). Scalability experiments up to 100 clients and drift-frequency ablations are convincing.\n\n+ .The paper is well-structured and visually clear, with informative figures (e.g., Figures 1–3) and theoretical justification of the extractor’s properties (R1–R5).\n\n+ The efficiency and practicality of FEROMA are good. FEROMA achieves low computation and communication overhead (profile size ≤ 0.35 % of model parameters). From the idea perspective, the adaptive similarity-based weighting and automatic aggregation mode selection are conceptually neat and practically beneficial.", "weaknesses": "- Writing quality is high, though a few sections are dense and could benefit from intuitive explanations or examples (e.g., DPE stochasticity (R3)). The paper includes an extensive set of experimental results across multiple datasets, baselines, and drift/shift scenarios, which demonstrates strong empirical effort and thorough evaluation. However, the presentation could be improved for readability. Currently, the dense amount of numerical results and tables makes it difficult for readers to extract the main insights quickly. I encourage the authors to streamline the narrative by emphasizing key findings in the main text (e.g., highlighting representative cases or summarizing trends) and moving secondary details to the appendix. Adding clearer transition sentences and summary paragraphs after major result sections would also enhance flow and help readers better connect the experiments to the paper’s main claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "H2tgGxwqLZ", "reviewer": "Reviewer_7n8b", "summary": "Existing federated learning (FL) methods often fail to address distribution shift across clients and distribution drift over time, or they rely on strong assumptions such as known number of client clusters and data heterogeneity types, which limits their generalizability.\nThis paper aims to develop a new FL solution that can handle distribution shift and drift without relying on client or cluster identity.\nThis method builds on client distribution profiles that guide model aggregation and test-time model assignment through adaptive similarity-based weighting.", "questions": "- Can the authors revise the paper to enhance the FEROMA’s dynamic aggregation using more solid theoretical analysis?\n- Would it be possible to make the FEROMA’s distribution in a way to be self-adaptive to evolving latent spaces?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The proposed method introduces a unified approach that can handle both distribution shift and distribution drift. It covers scenarios that previous FL methods treat separately. This generalization is meaningful and provides a comprehensive theoretical foundation for dynamic non-IID conditions. Further, the way that this paper designed Distribution- Profile Extractor (DPE) with explicit differential privacy guarantees, bounded stochasticity, and theoretical validation supports a strong methodological foundation. \nThe experiments utilize six, various non-IID scenarios, and drift frequencies, and large-scale client scenarios.", "weaknesses": "This reviewer has some concerns about weaknesses. \nThe effectiveness of proposed method depends on the quality of the DPE. However, DPE itself requires a pretrained model. The manuscript even states “the effectiveness of FEROMA similarly depends on the quality of its DPE, which must generate reliable representations of client data distributions”.\n Poor representation quality can undermine profile fidelity and mapping accuracy, and ultimately make the method less reliable in low-resource scenarios.\nThe paper does not clearly quantify the robustness of proposed method against unseen or out-of-support distributions.\nWhile the manuscript states the following, it does not clearly provide theoretical analysis of convergence guarantees or sensitivity of performance to hyperparameters:   “FEROMA dynamically selects the best aggregation strategy—ranging from clustered to personalized or global—based on client distribution profiles, and naturally extends to test-time adaptation without retraining.”,", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "O702Kk1fCn", "reviewer": "Reviewer_5gUD", "summary": "This paper presents FEROMA, a FL framework that attempts to handle both distribution shift and drift without relying on client or cluster identity. The claimed contributions include\n* a model aggregation strategy that adapts to client distribution profiles and can be extended to test-time adaptation without retraining\n* a framework that works under both distribution shift and drift with minimal communication and computation  overhead, which is comparable to FedAvg\n* theoretical bounds and empirical  measurements to validate the framework' practicality and privacy-preserving feature", "questions": "1. I don't understand how Global FL fallback works in Figure 3. But I guess, in the cases, $\\bar w_{t}^{2,j}$ is a 0-vector. But why $\\bar w_{t}^{k,j}\\approx 1/|A_{t-1}|$ instead of $\\bar w_{t}^{k,j}=1/|A_{t-1}|$?\n\n2. Why there are N/As in tables for different methods under different settings, for example (ATP, CheXpert), in Table 1? There are no explanations. \n\n3. Will the observation and claims still be similar if the communication round is set to 100 or even 200? Do all clients participate in all rounds?\n\n4. How the FEROMA compare to other test-time methods under the testing scenario mentioned in appendix D?\n\n5. Can the PCA map $g$ be computed on the server side? Since the random seed is fixed and the ${m^{-}, m^{+}}$ are the same for all clients, I don't see why the computation of $g$ needs to be done on the client side.\n\n\n**Minor Comments**\n1. The difference between solid and transparent blue/red lines are not explained.\n2. Equation (1) abuse the notations of weights to be optimize and the optimal solution. Similarly abuse use of notations also happen in R(1) when referring to the distribution over the dataset and $\\mathcal{A}$ to denote active clients and measurable set for DP definition. matrix of latent vectors $s^(k)_t$ and the sample size $s^{(k)}$.\n3. The notation $d_t'^{(k)} \\subseteq d_t^{(k)}$ does not make sense to me. Maybe the authors intend to say $d_t'^{(k)}$ is a subvector of $d_t^{(k)}$.\n4. Uncommon terms \"Lipschitz-equivalent\" is not properly defined. \n5. Figure 3. The weights for $\\{d_{t}^{(i)}\\}_{i=3}^{6}$ are all non-zero, which does not match the text \"...exactly one weight is nonzero,...\".\n6. Some necessary details are deferred to the the Appendices, and one needs to jump back and forth to locate the details.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* To my best knowledge, this is the first work that explicitly designed to address both distribution shift and distribution drift, during training as well as test time. [Disclaimer: (1) I have limited knowledge on the relevant literatures in the FL under distribution/concept drift. (2) Though the methods can be used for test-time adaptation, it's effectiveness is not fully justified.]\n\n* The presentation is mostly clear and easy to follow and the visualizations help to understand the proposed method. \n\n* The experiments and ablation are comprehensive in studying distribution shift and drift accross different datasets.", "weaknesses": "1. Usages of math notations should be correct and consistent. They should be used to deliver clear messages and supplement natural languages, instead of further creating confusion. Examples of the abuse of notations and incorrect usages can be found in minor comments, which I can infer the messages. However, there are cases where the inconsistent (or incorrect) usage of math notations making the understanding of the key algorithm designs, hence the contribution, difficult. For example, the notation $(x^{(k)}, y^{(k)})$ is first introduced in line 123, which is defined as the local dataset and said to be **fixed**. Later, in Definition 3.1, $(x^{(k)}, y^{(k)})$ is then re-defined as $(x^{(k)}, y^{(k)})\\sim P(X_t^{(k)}, Y_t^{(k)})$, making the $(x^{(k)}, y^{(k)})$ a random matrix, hence making $d_t^{(k)}$ a random vector. One may argue that $(x^{(k)}, y^{(k)})$ is deterministic and the randomness of $d_t^{(k)}$ comes from the fact that the $\\phi_\\psi$ is a random mapping. However, that's not the point. The point is that the readers should not guess what's the authors' actual messages. This further bothers me when attempting to understand the (R1) and (R3) right after the Definition 3.1. In (R3), it clearly states that $d_t^{(k)}$ is a \"random variable\" (actually a random vector), however in (R1), the way the equation is presented makes $d_{t_1}^{(k_1)}$ and $d_{t_2}^{(k_2)}$ look deterministic. Given $d_t^{(k)}$ is a \"random variable\", it only makes sense to me when the inequality is read as \"it holds with probability 1\" or \"it holds under the expectation with respect to distribution ...\". Since this part is not clear to me, I did NOT check any mathematical proofs.\n\n2. While understanding the page limitation, I still think it's not appropriate to omit the results on the test-time performance in the main paper and to make claims on the performance of FEROMA based on its performance on MNIST(table 2). For example, in table 2, the proposed method can easily show double digits gain over other methods nearly across the boards. However, in other tables the gain may not be significant or some baseline methods may have better performances, e.g., Table 14 (MNITS, APFL wins) , Table 18 (Cifar-10, results are mixed), and Table 24 (Cifar-100, FEDRC and FEDEM wins under low non-iid level). In contrast, Table 1 is more compelling to support some claims. It would be better if the authors could clarify what cases are used to compute the average numbers. Simply mentioning \"under varying drifting frequency, non-IID types and levels\" is not enough. \n\n\n3. Some parameters on the experiment setups are not justified or not mentioned. \n\t* Unjustified parameters: For example, the communication round is set to 20. This might be okay for datasets like MNIST since it's relatively easy, hence requiring less round to get decent performances. But for relative challenging datasets like cifar100, 20 rounds may not be sufficient. Will the observations and claims remain the same, when the communication rounds are set to 100 to allow all methods to converge? \n\n\t* not mentioned: In the problem formulation, authors introduced $\\mathcal{A}_{t}$ to denote active clients in round $t$. I would assume that only a fraction of client would participate in each round. But in the experiments, authors only mentioned \"number of clients in 10, 20, 50, and 100\". Does this mean all clients participate in all rounds?\n\n\nI am willing to raise the score if my concerns are properly addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)