A Federated Generalized Expectation-Maximization Algorithm for Mixture Models with an Unknown Number of Components
Authors: Michael Ibrahim
Nagi Gebraeel
Weijun Xie
Venue: ICLR 2026 Poster (2026)
Keywords: clustering
gaussian mixture model
federated learning
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We present a novel generalized expectation-maximization algorithm for federated clustering problems where the total number of unique clusters across clients is unknown.

Abstract: We study the problem of federated clustering when the total number of clusters $K$ across clients is unknown, and the clients have heterogeneous but potentially overlapping cluster sets in their local data. To that end, we develop FedGEM: a federated generalized expectation-maximization algorithm for the training of mixture models with an unknown number of components. Our proposed algorithm relies on each of the clients performing EM steps locally, and constructing an uncertainty set around the maximizer associated with each local component. The central server utilizes the uncertainty sets to learn potential cluster overlaps between clients, and infer the global number of clusters via closed-form computations. We perform a thorough theoretical study of our algorithm, presenting probabilistic convergence guarantees under common assumptions. Subsequently, we study the specific setting of isotropic GMMs, providing tractable, low-complexity computations to be performed by each client during each iteration of the algorithm, as well as rigorously verifying assumptions required for algorithm convergence. We perform various numerical experiments, where we empirically demonstrate that our proposed method achieves comparable performance to centralized EM, and that it outperforms various existing federated clustering methods.

Published as a conference paper at ICLR 2026
A FEDERATED GENERALIZED EXPECTATION- MAXIMIZATION ALGORITHM FOR MIXTURE MODELS WITH AN UNKNOWN NUMBER OF COMPONENTS
Michael Ibrahim, Nagi Gebracel & Weijun Xie
H. Milton Stewart School of Industrial and Systems Engineering
Georgia Institute of Technology Atlanta, GA 30332, USA (mi mnlbrahim411 ngebraeel3 3, wxie) @ggtech..eee edd
ABSTRACT
We study the problem of federated clustering when the total number of clusters K across clients is unknown, and the clients have heterogeneous but potentially over- lapping cluster sets in their local data. To that end, we develop FedGEM: feder- pnpectatioonmaiiizattio algorithm for the training of mixture
ated generalized models with an unknown number of components. Our proposed algorithm relies each of the clients performing EM steps locally, and constructing an uncer- tainty set around the maximizer associated with each local component. The cen- tral server utilizes the uncertainty sets to learn potential cluster overlaps between clients, and infer the global number of clusters via closed-form computations. We perform thorough theoretical study of our algorithm, presenting probabilistic convergence guarantees under common assumptions. Subsequently, we study the specific setting of isotropic GMMs, providing tractable, low-complexity compu- tations to be performed by each client during each iteration the algorithm, as well rigorously verifying assumptions required for algorithm convergence. We perform various numerical experiments, where we empirically demonstrate that our proposed method achieves comparable performance to centralized EM, and that it outperforms various existing federated clustering methods.
INTRODUCTION
Original equipment manufacturers (OEMs) of capital-intensive industrial systems, such as power generators and medical imaging systems, often enter into lucrative long-term service contracts (LTSCs) with their clients, guaranteeing adherence to stringent reliability standards. Failure to meet such guarantees can incur multi-milioo dollar penalties (Schimmoller 2001 Thompson et al. 2003). To manage these risks, OEMs must be able to accurately detect and diagnose faults in timely manner (Lei et af.] 2020 Dutta et al., 2023 Shuming et al.) 2024). However, OEMs face several critical challenges. First, OEMs do not have prior knowledge of all the possible fault classes Second, OEMs cannot rely solely on labels provided by their clients due to the absence of global labeling standard and differing maintenance practices, which result in inconsistent labels. Third, clients cannot readily share their raw data with the OEM due to the size and dimensionality of the data, and privacy concerns. Thus, centralized model training is infeasible.
Federated Learning (FL) (McMahan et al.) 2017: Konečný et al. (016)) offers a promising solution. However, the majority existing FI. efforts (Li al.) (2200) Wang et al., 2020 Karimireddy al., 2020 Arivazhagan et al.) 2019) Lee et al., 2023) assume that all clients share identical cluster sets and are primarily focused on supervised learning. Unfortunately, these assumptions are not compat- ible with our problem setting they violate one more of the critical challenges mentioned above.
Recent works on unsupervised FL (Dennis et al. 2021 Stallmann & Wilbik 2022 Garst & Rein- ders) 2024] Bárcena et al. 2024 Yfantis et al. (025) relax the assumption of identical cluster sets across clients. However, they still assume that the server knows the total number of unique clusters
in adyance- assumption that does not hold in our problem setting. While this assumption

---

Published as a conference paper at ICLR 2026
is relaxed by Zhang al. (2025). their algorithm has two critical limitations: (i) it requires clients to share arrays of the same size (i.e. dimensionality and cardinality) as the raw data, and (ii) client data can be easily reconstructed the central server via simple computations the information shared
by the clients, causing a violation of privacy.
This paper focuses on developing unsupervised federated learning methodology for dis- tributed clustering with an unknown number of clusters across privacy-constrainn clients with nigh-dimensional data. Our methodology enables central server (i) infer the total number of distinct clusters (components) that emerge across all clients without requiring access to raw data or prior knowledge of the cluster count, and (ii) determine the cluster memberships of each client.
Contributions. We introduce FedGEM: the first federated generalized expectation ponectation-mxxmizzziinn maximization (GEM) algorithm that can be used for the training of mixture models without prior knowledge of the global number of components. Our algorithm allows clients with overlapping clusters to collaborate on the training of cluster centers, whereas cluster weights are set locally each client. This allows for model personalization, where local cluster weights can adapt the global model to client-specific distributions. We summarize our main contributions next.
1. We develop the first federated GEM (FedGEM) algorithm for the training of mixture models with- out prior knowledge of the total number components. Our algorithm relies on uncertainty sets obtained by each client for each local component by solving an optimization problem. Intersections between the uncertainty sets enable the central server to detect cluster overlaps between clients via closed-form computations, allowing for collaborative model training.
2. We rigorously study the convergence properties of our algorithm and prove that iterates converge to neighborhood of the ground truth model parameters with a certain probability under common assumptions. This allows our algorithm to correctly estimate the true total number of unique clusters.
3. We examine various theoretical aspects of our proposed algorithm in the context of multi- component isotropic Gaussian Mixture Models (GMMs). To that end:
(a) We derive low-complexity, tractable, and bi-convex reformulation of the optimization prob- lem that is solved by the client to obtain the local uncertainty sets.
(b) We prove the first-order stability (FOS) condition for multi-component isotropic GMMs, which allows to study the contraction region and prove convergence of our proposed algorithm.
4. We perform a thorough empirical evaluation on popular and synthetic datasets, showing that our algorithm outperforms state-of-the-at ones while scaling well with problem size, at times even outperforming methods with prior knowledge of the cluster count. We also highlight our algorithm's 's strong performance various problem settings, including ones that violate modeling assumptions.
RELATED WORKS
Federated Learning. The canonical FL algorithm, FedAvg McMahan et al. 2017), is primarily designed for supervised deep learning. It aggregates model gradients across clients to train a sin- gle global model. However, it can perform poorly under non-lll) client data, often converging to suboptimal solutions. Numerous methods address this issue, including FedProx (Li et al. 2020). FedNova (Wang et al., 2020). SCAFFOLD (Karimireddy et al.] 2020). FedPer (Arivazhagan et al. 2019), and FeaL2P (Lee et al. 2023) However, these efforts overwhelmingly focus on supervised settings and assume that all clients have identical cluster sets their training data.
Several works have attempted to relax the common cluster set assumption. For example, FedEM (Marlog et al.) 202T) trains a global mixture model with localized component weights support personalization. A different version of FedEM is introduced by Dieuleveut et al. (202T). focusing on reducing client heterogeneity. Additionally, FedGMM (Wu et al.) (023) tackles covariate shift using Gaussian mixtures. However, these methods assume prior knowledge of the global number of components, making them unsuitable for real-world problems with unknown cluster counts.
Federated Clustering. Recent efforts have explored unsupervised federated clustering, allowing clients to have heterogeneous cluster sets. Examples such efforts include k-FED (Dennis et al. 2021). FFCM (Stallmann & Wilbik 2022). and Fe FedKmeans (Garst & Reinders 2024). among oth- ers [Bárcena et al. 2024]] Yfantis et al. (225).. However, these works still require prior knowledge of the global number of clusters, limiting their applicability in many real-world problems.
2

---

Published as a conference paper at ICLR 2026
To the best of our knowledge, only AFCL (Zhang et al. 2025 attempts federated clustering without requiring prior knowledge of the global cluster number. However, this work involves clients sharing arrays of the same size the local data. It also suffers from significant privacy vulnerabilities that allow data reconstruction at the server via simple scalar multiplication and subtraction operations.
Centralized Clustering with Unknown Cluster Number. A canonical example of such mod- els is the Dirichlet Process Gaussian Mixture Model (DP-GMM) (Antoniak) 1974), which extends GMMs by placing nonparametrie Dirichlet Process prior over the mixture components. Other approaches include the density-based DBSCAN (Ester et al. 1996), the scalable variational infer- ence approach for DP mixture models (Hughes Sudderth, 2013). and the neural network-based DeepDPM (Ronen et al. 2022). However, all of these methods assume centralized access to the full training dataset.
PROBLEM SETTING
We consider federated clustering problem with G clients and an unknown number K of total clusters (we use "cluster" and "component" interchangeably). Each client has access to Ng local data samples ( N - generated from a local mixture model M2(2) = _k,- (x, where Kg is the local number of clusters, and Pk,, (a= (0x)) are the independent component distribu- tions parameterized by ground truth parameters 0% and weighted by fixed The for all Kg E [Kg]. We denote the vectorized concatenation of all ground truth parameters at client by 0%. We as- sume that Kg is known for all clients g E [G]. whereas the global K is unknown. We also as- sume that clients may have some overlapping clusters, but no client has all the clusters locally, i.e.. 2 & Kg < K, Vg E [G].
We denote the minimum and maximum distances between any two unique ground truth cluster parameters by Rmin and Rmax respectively. That is Rmin = min..je(x].++] 0 021le and Rmax = max.,ee(K),ts | 02 These quantities are used to study the convergence behavior of our algorithm and do not need to be known in advance to use our algorithm. We make the following crucial assumption to support algorithmic convergence analysis (in Section
Assumption (Ground Truth Parameters). Each global cluster à E [K] is parameterized by fixed ground truth 0% that is consistent across all clients where the cluster is present. However, the weight assigned to the cluster may vary locally across clients.
Remark 1. Assumption ! motivates our algorithm design, where clients with overlapping clus- ters collaborate learning the shared cluster parameters while retaining personalized cluster weights. This respects the non-lID nature of the federated data while enabling collaborative training.
At client g, we denote the local population expected complete-dan data log- log-liiel by
K
(0,00)) = M(=))
Σ
Th, (x, O²) log( Tk, Pk, (x))),))
where Tik, (2, (x, 0)) is the posterior responsibility function of the kthh component, computed using current parameters 0' Similarly, we denote the local simite-sample expected complete-data log- likelihood function by
N K
(,(00)))) = Ng my h
Ykg (Zng. O3) log(Tk,,P,, (2n, 0kx ))-
4 OVERVIEW OF FEDGEM ALGORITHM
Our proposed FedGEM algorithm consists of two stages: (i) an iterative collaborative training stage, and (ii) single-step final aggregation stage. (Pseudo-code in Algorithm in Appendix A.I]
The collaborative training stage can be summarized as follows. Client: (i) performs (potentially multiple) EM steps locally, (ii) solves optimization problem obtain the radius of uncertainty
3

---

Published as a conference paper at ICLR 2026
set for each component centered at its corresponding maximizer of (,(0011%) and (iii) broadcasts the maximizer and uncertainty set radius pair for each component to the server. Server: performs aggregation using overlaps between uncertainty sets and re- re-broadcasts updates clients.
In the final aggregation step, the server merges cluster estimates from different clients if they are within specific radius of each other. This enables the server to estimate the total number of unique global clusters and determine the cluster membership of each client. Before discussing our algo- rithm, we make the following assumption, which can be verified for common models such GMM. Assumption 2 (Strong Concavity). Each of the Kg terms in the population 2g(0,10g) or finite- sample Qg (0,00')) client ! are strongly concave in 0g for all g E [G].
4.1 CLIENT COMPUTATIONS
Each client performs two vital tasks during each iteration t of our algorithm: (i) it performs (potentially multiple) EM steps given current model parameters og-"), and (ii) it solves for the
(#) radius Ek2 of the uncertainty set ut,) associated with the maximizer of each local component kg- Firstly, we examine the EM steps, which are displayed next.
Pk, (Dn, 10 Ko K2 20 Ti Pk, (nn, (0(1))
E-step:
7k (2n, 1)
Vkg [Kg], Vng e [Ng] (1)
M-step:
Me, (0,(0)))1)
arg max 2xg ERd
Log(TTK, Pkg (k.)) Vkg [Ky]
Next, client 9 solves for an uncertainty 24 to capture potential perturbations in Mr (0)
associated with each component kg E [Kg]- This uncertainty set is defined as a Euclidean ball B2( (0(--1) /ell
(()) centered at Mr, (0(-)1) and whose radius is fek We construct the
uncertainty such that any iterate m.x (0(--1) (Mr, (0(- finite-sample expected complete-dee log-likelln likelihood function from the previous iteration. That is
does not decrease the
nk, (Zn, ((--1)) log( Th (2n, mix (6 (((-)))))
Tky (iny (((-1)) log(TH, TT. Pk., (an, (ān,10f,, (0 1))
10g
This renders our proposed algorithm an instance GEM, allowing it to exhibit similar conver- gence behavior locally at client g to the EM algorithm
show later. Client g may obtain the radius / V=l² of the component's 's ky uncertainty set by solving the following optimization problem,
which admits a unique solution as we argue Proposition
Jk, (0)-1))
max
Ekg
Ekg
t.
Σ
Th, (#n, 1((-1)) log(TE, PK, (Bm, mix (0((-1))))
=1=1
N >
7kg (Zng: (((-1)) log( PK, (2.,, ((()-))))) Vmee (0(t-1)) B2
(0 (0(t- Ek,))
Proposition 1 (Local Uncertainty Set Radius Problem). Suppose Assumption must exist a unique solution εk, 2 kg [Kg] and all clients g [G]. (Proof in Appendix C.1
2 holds. Then, there
the optimization problem Js, Jx,(0' (0(t-1) for all components
4

---

Published as a conference paper at ICLR 2026
After completing local computations, each client y transmits tuple (Mx, (0 ( of the ob- tained local maximizer and uncertainty set radius for component kg E [K,] to the central server. This, however, only applies in the collaborative training stage. During the final aggregation step, each client transmits tuple (Ms, (0(--)) &R, (imal) the central server, where Ek, is the final aggre-
gation radius for component hig and is treated as user-defined hyperparameter.
4.2 SERVER COMPUTATIONS
In both collaborative training and final aggregation stages, the server uses the uncertainty sets 2 Vkg [Kg] Vgg [G] to identify cluster overlaps between clients. This allows the server to group clients' components into super-clusters via pairwise comparisons and series of closed- form computations. Specifically, the server begins by initializing an estimate R(t) = 0. It then checks if the uncertainty sets of components ki at client g and kyg at client g' overlap. That is, it checks if: MM, (0!-")) Ma, (0 (0)="))ll= "> Vo + Veg.
(8) If this holds, then the server groups the two components kig and hig into a single super-cluster. Consequently, if one or both of the com- ponents already belong to a super-cluster, the server performs super-cluster merge and updates RO), there is no overlap, the components are assigned to different super-clusters, and K-(1) is updated accordingly. This repeats for all ky E [Kg]] and kg [Ky] g'] at all clients 9, g [G].
(1)
During the collaborative training stage, the server relies on uncertainty set intersections to compute an updated parameter vector o for component kg at client y. This updated vector remains within its respective uncertainty set, thereby facilitating convergence. This is achieved by initializing set T of vectors containing only the estimate (T,,(O(())) for each component ki at client g- 9
Subsequently, if any intersections are found between " and any other 4 for any g' E [G]]g
then an optimal vector n is added both the sets Tx(1) and T,(t) This vector D can be written as:
(8)
v" = "M,,,, -M, + clip 0.5,
R
Fkp
(0 - My (o 1)
w
where w = MMM (0 - My, >1- and the clip(æ, a, b) function limits the input æ to the range [a, b]. After all comparisons are complete, the server obtains the updated parameters o!!) for component kg at client g by aggregating all the vectors in set T
In contrast, in the final aggregation step, the server aggregates all the estimates Mr (0((-1)) of components kg that belong in the same super-cluster. This ensures that clients eventually reach consensus on the parameters of shared clusters. We present the server computations pscudo-code in Appendix A.2 We also present a method for potentially improving the efficiency of the server computations and analysis of communication costs incurred by algorithm in Appendix B.5
4.3 CONVERGENCE ANALYSIS
We provide a convergence analysis for our algorithm in the finite-sample setting. This is built upon population convergence analysis, which provide in Appendix B. The idea in convergence proofs to show that an algorithm for component Kg at client g whose iterates are m, (0 (0(
B2 Mr, (0 "):y' converges at a desirable rate to some neighborhood of the true parameters ot, This ensures that estimates of the same component from different clients can eventually be aggregated due to their proximity at convergence. Our convergence analysis relies on the FOS property introduced by BBlakrishhan et al. (2014), which is defined next. Subsequently, we provide key technical assumptions, followed by our convergence results.
Definition 1 (First-Order Stability). The expected complete-data log- ligeeiil function Q(-10) is said to obey first-order stability with parameter ,3 if for any 0 By ( a) Ak e [K] we have that
[VQ()(())))) - VQ(M(0))**) [|2
0 0 0*1 ||2,
5

---

Published as a conference paper at ICLR 2026
where B E R is constant, and 0 is the vectorized concatenation of all 0s for all & [K]. Assumption 3 (First-Order Stability). The expected complete-data log-likelihood Qo (000000 at client obeys the FOS condition with parameter Bg. such that 0 & Bg < X for all E [G]. where Ag is the strong concavity parameter of Qg(0,)
Assumption 4 (Continuity). The local population and finite-sample Qg(0 and (gg0),)) respectively, at client g are continuous in both of their arguments.
Assumption 5 (Likelihood Boundedness). The local population and finite-sampei true log- likelihood functions Lg (0g) and Ei (02), respectively, at client g are bounded from above.
Assumption 6 (Finite-Sample and Population M-Step Proximity). Let & I B2 kgg 0g), and cg
unit (Ng: 5g) $ sup Mr, (0,)) MR, (02))l2 < unif (Ng. 5g). 0e
Lg )ag be some constant. Then, with probability (w.p.) at least (1 5g), xg
where Mx, (6,)) is the M-step map associated with the population Qg (g(0)
Remark Assumptions 3] are standard assumptions that are commonly utilized in works focused on EM algorithms such as BBlakrishhan et al. (2014) Yan et al. [017]] Marfoq et al., (021), and are
verifiable for isotropic GMMs as we show later.
As shown by [Balakrishnan et al. (2014). 4), if Assumption 6 holds and the population M-step iterates converge as described in Appendix B.I then the finite-sample M-step iterates converge to neigh- borhood of the true component parameters tx, w.p. at least (1- We express this mathematically next, followed by Theorem
T asserting convergence to a single point rather than oscillating.
Mr, (((--1)))) -
Bg
1) 0%
1l2
1
8y 9 (Ng, 5g)
vell (t-1)) e B2 :2 :g)): (5)
Xg
1
where 0g is the radius of the contractive region associated with the population EM iterates.
Theorem 1 (Single-Point EM Convergence). Suppose Assumptions 7 through o hold, and that
0(-") B2 ag). Then the finite sample EM iterates Mr, (0)-1)) converge single point
within radius 2e (N 0g) from the ground truth parameters kk. (Proof in Appendix C.2
Now, consider a local sinite-sample GEM algorithm whose update during each iteration t any mig (o(-))) B2 (Mx, (o!-))); &k
(0)) where the radius v Sk is obtained by solving the problem in BD. We show in Theorem 2 next that this algorithm exhibits very similar convergence behavior to that shown in (5). Subsequently, we show in Theorem 3 that our proposed FedGEM algorithm infers
[0)
the true global number of clusters K with certain probability.
Theorem 2 (Local Convergence of Finite-Sample GEM). Suppose Assumptions 7 through 0
hold.
Consider a GEM algorithm whose iterate fi, fok,00) (0t iteration is such that fiu (0( (t) B2 MR, (0 (0!--)); )iN'' ekg) where the radius v Skg is obtained by solving the problem in 3 Then,
g
this algorithm converges to a neighborhood of the ground truth parameters 0%, as follows:
||, (0 - 12,
B Ag
k 0% 1l2l
Bo 5g) +E(t) vok- (--1)) B2( ;ag),
1
X
w.p. at least dg. with 2(L) MR, (o((-1))
/ 0!!-"1112 0 as 1. cc. (Proof in Appendix C.3
Theorem (Number of Clusters Inference). Suppose that all the assumptions associated with The- orem hold. and that |M, (0(--1)) " diminishes to 0 at sufficiently fast rate. Suppose Jinal
further that the final uggregation radius ex, at client g is set such that εk, = eggi ( 5g), and Sinal Bas Then, the final R inferred by the FedGEM algorithm is K2
Jinal
such that (CG) Ek2
equivalent to the true K w.p. at least [[ 1k 1101 5g). (Proof in Appendix C.4

---

Published as a conference paper at ICLR 2026
5 FEDGEM FOR MULTI-COMPONENT ISOTROPIC GMMS
Model Setup. Now that we have introduced our FedGEM algorithm, and studied its conver- gence in a general sense, we examine it in the context of an isotropic GMM. More specifi- cally, we consider the setting where each client g data is governed by a local mixture model (MM)( = L _k Tk, ((x) Ia), where ((z) Ia) is the Gaussian density with identity covariance. We denote the minimum and maximum component weights at client by Tming and Fimng The population and finite-sample
7 respectively. Moreover, denote the ratio Kig
Ruing
M-steps associated with this model admit the following closed-form solutions.
EB-CMM, x O' )z] E. Ex GMM (7k, (x., 0,)]
Population M-step:
Mag (0')
Whg E [Kg].
(6a)
En The BBy. 0 Fing
N₂
Finite-sample M-step:
Mr,(0%
Vkg [Kg]-
(6b)
Eng Tkg (Zng 6%)
Note that for the described model, the population and finite- samile expected complete-data log- likelihood functions Q, (0,10,) and (0, (',,)'')), respectively, are strongly concave in 0, and contin- uous in both of their arguments. Moreover, if 6 = 6% the strong concavity parameter of Qg (0 ( Tmin, Finally, the true population and sampe log-likelihood functions associated with this model are bounded from above due to the identity covariances and fixed weights for all components.
As shown in (6b), the finite-sample M- M-step associated with our model admits a closed form. There- fore, it remains to derive a tractable reformulation of the uncertainty set radius problem in (3). Next, we introduce Theorem where we derive a bi-convex, 2-dimensional reformulation of the problem in B Additionally, we introduce solution Algorithm 4 in Appendix B.2 to solve the problem, accompanied by Proposition 2 in Appendix B.2 asserting that the algorithm enjoys a low worst- case time complexity. Finally, provide preliminary differential privacy discussion for the sample maximizers shared by the clients in Appendix[B.4]
Theorem 4 (Radius Problem Reformulation). The semi-infinite optimization problem Jr, (0' in B admits the following tractable, bi-convex, 2-dimensional reformulation for the isotropic GMM described in this section. (Proof in Appendix C.5
Jk, (00)) =
max Ekg MOgEE ER
Ckb
N2 Σ 19g
t.
Eks ai +
Ykg (2ng. 0²r) (Issn,
(0,))1 -113n, .8%,11 112 Ekg) axa +
N₂
No Σ ny
Σ
nkg (Zng. 0'))
7h (ān,
0z)1 0% 112 <0
N Okk Jk, ( )²)
Convergence Analysis. To guarantee the convergence of our FedGEM algorithm for the multi- component isotropic GMM, we verify three key properties. Namely, we present Theorems 6 7 g).
and 8 in Appendix B.3 to establish the FOS property of (,,0,,,)), study the conditions under which Ms, (0g,) is contractive, and establish probabilistic upper bound on the distance between the population MR, (0,) and the finite-sample Mr, (0y). respectively. Consequently, the convergence of our algorithm for isotropic GMMs follows from these results. However, these results require the clusters to be well-separated.
7

---

Published as a conference paper at ICLR 2026
6 NUMERICAL EXPERIMENTS
We present a comprehensive set of numerical experiments that benchmark the performance of our proposed FedGEM algorithm against leading state-of-the-arr federated clustering methods. All num- bers reported in this section are averaged over 50 repetitions. Confidence intervals and error bars represent one standard deviation. Randomness in repetitions arises from initialization, cluster signments, and data Shuffing//enerraioon over, weigh each client g by its sample count Ng. and we set the final aggregation radius equiva- lently for all clients. In all experiments, clusters are assigned randomly to the clients, whereby each client, g has Kg clusters such that 2 E Kg K. More experimental details and results are provided in Appendix(D) D Additionally, a Scalability Study is provided in Appendix demonstrating that our algorithm scales exceptionally well with problem size compared to relevant benchmarks. All code associated with this paper is available at httpt:tt/////......tt httpp:///ggttuue com/ wwwraaiimm//eeeaaeee
We assign equal T for all kg E [Kg] at client g. More-
Our Method. Our method the isotropic GMM from Section 5 trained via our FedGEM algorithm.
Evaluation Metries. We utilize the Silhouette Score (SS) |Rousseeww 1987) for hyperparameter tuning as it does not require label knowledge. However, we mainly rely on the Adjusted Rand Index (ARI) (Hubert & Arabie, 1985) to evaluate model performance as it is robust to cluster shape and size unlike other metries. We report experimental results in SS in Appendix D.4
Hyperparameters. We tune the final aggregation radius for FedGEM via cross validation. However, do not directly tune the radius itself. Instead, we use the heuristic =fimaal Vyglmin where Rmin, ",yN," the minimum distance between any two estimated cluster centroids at client g. and Ug is the hyperparameter we tune (set equivalently across all clients for simplicity). This heuristic allows the final aggregation radius to scale with the feature space and the number of samples available each client. We provide a thorough discussion on hyperparameter tuning is provided in Appendix D.3] We set hyperparameters for the benchmark methods as described in their associated works.
6.1 BENCHMARKING
This study aims to compare the performance of our proposed method to that of various existing methods using an array of popular benchmark datasets.
Datasets. We use MNIST, Fashion MNIST (FMNIST), Extended MNIST (EMNIST), CIFAR-10, and other datasets from the UCI repository. For all datasets, we use 70% of the samples for training and the rest for testing, except for EMNIST where we use 50% of the samples for training. We use G = 100 for MNIST, FMNIST, and CIFAR- G = 25 for EMNIST, and G for UCI datasets.
Baselines. We compare our method to 2 centralized and 5 federated clustering methods from the literature. The centralized ones are a GMM and a DP-GMM (Antoniak 1974). The federated methods are k-FED (Dennis et al. 2021). FFCM-avgl and FFCM-avg2 (Stallmann & Wilbik 2022), > FedKmeans (Garst & Reinders 2024). and AFCL (Zang et al., (025). Note that DP-GMM and AFCL are the only benchmarks that do not assume prior knowledge of K.
Results. The ARI attained by all models and the estimated number of clusters estimated by models with unknown K are shown in Tables outperforms AFCL for all datasets, which is the only other federated clustering model with unknown K. Additionally, our model also consistently outperforms DP-GMM, which can be attributed to its more accurate number of cluster estimation. Another key observation is that our method even outperforms some clustering algorithms with known K in various datasets. This result underscores the significant practical impact of our proposed method, which does not require prior knowledge of K. We highlight that similar trends are observed when performance is evaluated via the SS as shown in Appendix D.4] emphasizing the impact of our model. Despite our model's strong performance, we also observe that it largely overestimates the number clusters in datasets such as CIFAR- 10, Frog A, and Frog B. While its estimate is the best achieved out of the models compared, this can potentially be further improved future work by examining more complex mixture models. Finally, we note that the datasets used are verifiably non-Gaussian via -Zikler multivariate normality test, and likely include cluster overlaps. This demonstrates that our model can perform well in practice even when assumptions are violated.
respectively. We observe that our method consistently
8

---

Published as a conference paper at ICLR 2026
Table 1: ARI attained by all methods on tested datasets. (Bold best, underline second best.)
Model
Known K?
MNIST
FMNIST
EMNIST
CIFAR 10
Abalone
Frog A
Frog B
Waveform
.287
.385
.235
.422
.966
447
.418
.262
GMM (central)
Yes
+. ..067
+.023
+.010
+.022
+ E.028
+.097
1.2
+ 0.1
.344
.288
.223
.358
1000
.617
.467
.27
Yes
= 2,82
+.101
t.031
+.043
t. 030
f.114
111
+ 0.0
FFCM-avgl
Yes
+ 0031
+.030
.164
+.007
+.035
4.029
+.094
+ 112
.442
+.029
.254
.336
.352
.268
HFCM-avg2
Yes
= 0.53
+.038
+.111
+.028
t.032
+.14
117
+.0
FedKmeeass
Yes
.640
.449
285
437
.998
.546
.492
.260
1.035
1.027
1.007
.204
1.033
1.143
1.118
1.015
DP-GMM (central)
No
.223
.255
=.021
+.011
+.013
+.003
t +.023
+.090
=.065
+.015
.038
.035
.034
.344
.272
.157
AFCL
No
i.002
+.002
+.005
+.002
1.020
f.108
2.079
+30
FedGIM (ours)
No
.452
.287
.285
.286
.138
.552
.468
.335
+.05
+202
+303
+.056
129
117
+.078
Table Estimated number clusters for algorithms with unknown K.
Model
MNIST
FMNIST
EMNIST
CIFAR-I0 10
Abalane
Frog A
Frog
Waveform
K
3
110.20
86.34
247.40
364.03
15.46
29.46
24.38
6.84
DP DP-GMM (central)
+66009
#5.61
423.46
+12.77
+2.20
+5.49
+:4.24
+0065
501 82
501.37
375.39
502.97
25.00
28.12
23.96
12.52
AFCL
122.51
119.70
16.42
14.26
11.11
13.63
17.59
58.67
37.72
12.14
23.94
20,00
4.42
FedGEM (ours)
113.60
13.65
19.99
17.07
11.40
6.2 SENSITIVITY
This study evaluates the performance of our model as Rmin changes. This includes non-well- separated settings, which violate the convergence conditions of the GMM Section We also examine the sensitivity of our algorithm to its hyperparameter in Appendix D.6
Dataset. The data used for this experiment isotropic Gaussian clusters generated via the make make_blok blobs module in Python. We control Rmin- requiring that the centers of at least two clusters in each dataset be Amin apart. Morcover, we study three key settings: i) nominal: data balanced across clients and clusters, ii) client imbalance: the data distribution across clients is [40%, 24%, 16%, 16%, 4%], and iii) cluster imbalance: the local data for each client is randomly distributed across the local clusters. For all settings we G Nunin 2500, and N 5000.
Baseline. We compare our model to a centralized GMM trained via EM as the latter represents a strong benchmark. This allows quantify the effects of our model's federation and unknown K.
Result. Figure T illustrates that the performance for both models improves as Rmin increases. No- tably, our proposed model achieves very close performance the centralized GMM, and even out- performs it with Rmin 2), This can be attributed to cluster heterogeneity across clients. That is, each client only has subset of the total clusters, so each client's clustering problem is poten- tially easier than the centralized problem. This can cause each client to perform better individually than the centralized model, which is trained on all clusters. Indeed, the benefits gained from cluster heterogeneity in clustering problems were first observed by Dennis et al. (2021).
worth noting that some of the settings explored this study involve overlapping clusters, violating the well-separated cluster assumption required for convergence. However, our model still achieves very competitive performance. This suggests that even when some assumptions are vio- lated, our algorithm can still converge in practice to well-performing model. Finally, observe that model's estimate k across all experimental settings is very close the true number of

---

Published as a conference paper at ICLR 2026
3003
0.0
03
0.5
3.55
0.85
0.0
01
03
(a) Adjusted rand index for our proposed FedGEM a centralized GMM trained via EM.
Nomuml
Clart inbebree
Chatc: Infolase
Stnter Chario
Stmtor Cranes d
15
(b) Number of clusters estimated by proposed FedGEM vs. the true number of clusters.
Figure 1: Results of the sensitivity study.
clusters K. While it tends to overestimate slightly in most settings, performance could potentially be further improved through better tuning of the final aggregation radius hyperparameter.
CONCLUSIONS
We introduce FedGEM: a federated GEM algorithm for training mixture models with an unknown number of components, geared towards federated clustering for clients whose local cluster sets are heterogeneous but potentially overlapping. Our algorithm requires clients to perform local EM steps, and compute uncertainty set centered at the maximizer corresponding to each component. These uncertainty sets are then shared with the server. The server leverages uncertainty set intersections to infer overlap between clients' clusters, allowing it to perform model parameter aggregation and to estimate the total number of unique clusters. We study theoretical aspects of our algorithm, where we prove probabilistic convergence under standard assumptions. Subsequently, in the context of isotropic GMMs. To that end, we derive a tractable and bi-convex reformulation of the problem used by each client to obtain the uncertainty sets, and we verify key assumptions re- quired to prove convergence. We empirically demonstrate that our proposed algorithm outperforms existing ones through series of numerical experiments utilizing synthetic and popular datasets. We provide thorough discussion on limitations and future work in Appendix F.3
study our algorithm
ACKNOWLEDGMENTS
This work was funded by the National Aeronautics and Space Administration (NASA), Space Technology Research Institute (STRI) Habitats Optimized for Missions of Exploration (HOME) 'SmartHab' Project (grant No. MONSSC19K1052) Weijun Xie supported in part by National Science Foundation grant 14,,444, and by the Office of Naval Research grant N00014-24-1-22
We thank the reviewers and the AC for their valuable input, which allowed us improve the pre- sentation and content of our paper.
ETHICS STATEMENT
All software and datasets utilized in this work are used under proper licenses, as detailed in Appendix D We do not release any data as part of our submission, and we provide full references all datasets used.
REPRODUCIBILITY STATEMENT
We have taken various steps to ensure the ease of reproducibility of both the theoretical and exper- imental aspects of this paper. On the theoretical side, we have provided full formal and complete proofs for all theoretical results, well as all the required assumptions and full description of
10

---

Published as a conference paper at ICLR 2026
the problem setting. More specifically, the detailed description of the problem setting is provided in Section [3] whereas the required assumptions are provided in throughout the main body of the paper. Additionally, supplementary theoretical results along with their formal proofs provided in Ap- pendix B The proofs of all theoretical results presented the main body of the paper are provided in Appendix [ Finally, detailed explanations and interpretations of all assumptions and theoretical results are provided in Appendix [ On the experimental side, we have provided a summarized de- scription of our experimental settings and results in Section 6 whereas we provided full detail on all aspects of the experiments as well as supplementary results In Appendices D and E] This includes all software and hardware details, all dataset license information and preprocessing details, well as hyperparameter details. Finally, we have made all the code used to run our experiments along with detailed instructions available at hthttpss//githhh com má hww.mmibbbmmmmmmmmmmmmmmmmmm. ///e////FF
REFERENCES
Charles E Antoniak. Mixtures of dirichlet processes with applications to bayesian nonparametric problems. The annals of statistics, pp. 1152-1174, 1974.
Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary. Fed- erated learning with personalization layers. arXiv preprint arXiv:1912.00818. 2019. 2019.
Sivaraman Balakrishnan, Martin J. Wainwright, and Bin Yu. Statistical guarantees for the em algo- rithm: From population to sample-based analysis, 2014.
José Luis Corcuera Bárcena, Francesco Marcelloni, Alessandro Renda, Alessio Bechini, and Pietro Ducange. Federated c-means and fuzzy e-means clustering algorithms for horizontally and verti- cally partitioned data. IEEE Transactions on Artificial Intelligence. 5(12):6426- 6441, 2024.
Jon Louis Bentley. Multidimensional binary search trees used for associative searching. Communi- cations of the ACM, 18(9):509-55 1975.
Freddie Bickford Smith, Adam Foster, and Tom Rainforth. Making better use of unlabelled data in Bayesian active learning. International Conference Artificial Intelligence and Statistics, 2024a.
Freddie Bickford Smith, Adam Foster, and Tom Rainforth. Making better use of unlabelled data in Bayesian active learning. International Conference on Artificial Intelligence and Statisties, 2024b.
Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
L. Breiman and C.J. Stone. Waveform Database Generator (Version I). UCI Machine Learning Repository, 1984. DOI:
Gregory Cohen, Saced Afshar, Jonathan Tapson, and Andre Van Schaik. Emnist: Extending mnist to handwritten letters. In 2017 international joint conference on neural networks (IJCNN, pp- 2921 2926.. IEEE, 2017.
Juan Colonna, Eduardo Nakamura, Marco Cristo, and Marcelo Gordo. Anuran Calls (MFCCs). UCI Machine Learning Repository, 2015. DOI: httss//dbb.or///222433/C/CCCCC
Don Kurian Dennis, Tian Li, and Virginia Smith. Heterogeneity for the win: One-shot federated clustering. In International conference on machine learning, pp. 2611-2621 PMLR, 2021.
Aymeric Dieuleveut, Gersende Fort, Eric Moulines, and Geneviève Robin. Federated-em with het- erogeneity mitigation and variance reduction. Advances in Neural Information Processing Sys- tems, 14:29553-22556 2021.
Nabanita Dutta, Palanisamy Kaliannan, and Paramasivam Shanmugam. Svm algorithm for vibration fault diagnosis centrifugal pump. Intelligent Automation & Soft Computing, 35(3), 2023.
Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and trends@ in theoretical computer science, 9(3-4)2211453 2014.
11

---

Published as a conference paper at ICLR 2026
Martin Ester, Hans-Peter Kriegel, Jöng Sander, and Xiaowei Xu. A density-based basted algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second Inter- national Conference Knowledge Discovery and Data Mining, KDD'96, 96, pp. 226-231. AAAI Press, 1996.
Jerome H Friedman, Jon Louis Bentley, and Raphael Ari Finkel. An algorithm for finding best matches logarithmic expected time. ACM Transactions on Mathematical Software (TOMS), (3):209-226, 1977.
Swier Garst and Marcel Reinders. Federated k-means clustering. In International Conference on Pattern Recognition, pp. 107-122. Springer, 2024.
Lawrence Hubert and Phipps Arabie. Comparing partitions. Journal of classification, 2:193-218, 1985.
Michael C Hughes and Erik Sudderth. Memoized online variational inference for dirichlet process mixture models. Advances in neural information processing systems, 26, 2013.
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International conference machine learning, pp. 5132-5143, PMLR, 2020.
Jakub Konečný, H Brendan McMahan, Daniel Ramage, and Peter Richtárik. Federated optimization: Distributed machine learning for on-device intelligence. arXiv preprint vrXiv:1610.02577 2016.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images, 2009.
Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online). Available: http://yann.eecu http://www..e....o///ee//m/iiiis 2010.
Royson Lee, Minyoung Kim, Da Li, Xinchi Qiu, Timothy Hospedales, Ferenc Huszár, and Nicholas Lane. Fedl2p: Federated learning to personalize. Advances Neural Information Processing Systems, 36: 36:14818 18-1888-4666 2023.
Yaguo Lei, Bin Yang. Xinwei Jiang, Feng Jia, Naipeng Li, and Asoke K. Nandi. Applications of machine learning to machine fault diagnosis: A review and roadmap. Mechanical systems and signal processing, 138:106587, 2020.
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and sys- tems, 2:429-450, 2020.
Othmane Marfoq, Giovanni Neglia, Aurélien Bellet, Lactitia Kameni. and Richard Vidal. Federated multi-task learning under a mixture of distributions. Advances in neural information processing systems, 34:1 13434-154417 2021.
Brendan McMahan, Fider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication--fficcientt learning of deep networks from decentralized data. In Artificial intelli- gence and statistics, pp. 1273-1282. Pmir, 2017.
Warwick Nash, Tracy Sellers, Simon Talbot, Andrew Cawthorn, and Wes Ford. Abalone. UCI Machine Learning Repository, 1994. DOI: http:///bb.org/10.444/////S
Yurii Nesterov and Arkadii Nemirovskii. Interior-Point Polynomial Algorithms in Convex Program- ming. Society for Industrial and Applied Mathematics, 1994. doi: 10.1
1970791.
Meitar Ronen, Shahaf E Finder, and Oren Freifeld. Deepdpm: Deep clustering with unknown number of clusters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9861-9870, 2022.
Peter Rousseeuw, Silhouettes: a graphical aid to the interpretation and validation of cluster analy- sis. Journal of computational and applied mathematics, 20:53-65. 65, 1987.
12

---

Published as a conference paper at ICLR 2026
Brian K Schimmoller. Long-term service agreements: Weighing the risks and rewards. Power Engineering, 105(8):28-22 28, 2001.
Yang Shuming, Xie Changlin, Cheng Yuqiang, Wang Biao, Ma Xunyi, and Wang Zinuo. Data-driven fault diagnosis for rolling bearings based machine learning and multisensor information fusion. IEEE Sensors Journal, 25(2):345233444 2024.
Morris Stallmann and Anna Wilbik. Towards federated clustering: A federated fuzzy c-means algo- rithm (ffem). arXiv preprint arXiv:2201.00 0, 2022.
Richard Thompson, Jason B Yost, et al. Long-term service agreements: Top 10 contractual pitfalls and how avoid them-part i.(gas turbines). Power Engineering, 107(2):56-59, 2003.
Jianyu Wang. Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems, 33:7611-7623. 2020.
Jeff Wu. On the convergence properties of the em algorithm. The Annals of statistics, pp- 95-103, 1983.
Yue Wu, Shuaicheng Zhang, Wenchao Yu, Yanchi Liu, Quanquan Gu. Dawei Zhou, Haifeng Chen, and Wei Cheng. Personalized federated learning under mixture distributions. In International Conference Machine Learning, pp. 77860-37873 PMLR, 2023.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. CoRR, NDV1700.007477 2017. URL httt:://pppp///////////////// org/
abs/1708.r 07747
Bowei Yan, Mingzhang Yin, and Purnamrita Sarkar. Convergence gradient em on multi- component mixture of gaussians. Advances in Neural Information Processing Systems, 30, 2017.
Vassilios Yfantis, Achim Wagner, and Martin Ruskowski. Federated k-means clustering via dual decomppoittion-based distributed optimization. Franklin Open, 10: 10:100204 2025.
Yunfan Zhang, Yigun Zhang, Yang Lu, Mengke Li, Xi Chen, and Yiu-ming Cheung. Asynchronous federated clustering with unknown number of clusters. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 22695-22703, 2025.
13

---

Published as a conference paper at ICLR 2026
A FEDERATED GENERALIZED EXPECTATION- MAXIMIZATION ALGORITHM FOR MIXTURE MODELS WITH AN UNKNOWN NUMBER OF COMPONENTS (APPENDICES)
CONTENTS
Supplementary Pseudo-code
16
FedGiit Pscuo-code
16
A.2
Detailed Server Computations Pscudo-code
17
B
Supplementary Theoretical Results and Analysis
19
Population Convergence Analysis for FedGEM Algorithm
19
B
Solution Algorithm for Client Radius Problem Reformulation for Isotropic GMMs
20
B
Supplementary Theorems Verilying Assumptions for Isotropic GMMs
21
B.4
Preliminary Differential Privacy Discussion for Isotropic GMMS
33
B.S
Computational Efficiency and Communication Costs
35
B.5.1
Improving Efficiency Server Computations
35
B.5.2
Note on Communication Costs
35
Proofs
35
Proof of Proposition
................................
35
Proof of Theorem
36
Proof of Theorem
36
4
Proof of Theorem
36
5
Proof of Theorem|4
37
Supplementary Experimental Details and Results
39
D.I Software and Hardware Details
39
D22 Datasets Unilized
40
D.2.1 Benchmarking Study
40
D.2.2 Sensitivity Study
................................
40
D33
Hyperparameter Details and Performance Evaluation
D.4 Supplementary Benchmarking Study Results
41
D4.1 Silhouette Score
41
D.4.2 Benchmarking Against Models with Known K via Estimation Heuristic
42
D.5
Supplementary Sensitivity Study Results
43
D.6 Sensitivity to flpeeraarameter
Scalability Study
44
Scalability on Image Datasets
45
Scalability on Synthetic Data
45
14

---

Published as a conference paper at ICLR 2026
F
Further Discussion
47
Justification and Interpretation of Modeling Assumptions
47
F2 Interpretation of Theoretical Results
47
Limitations and Future Work
48
15

---

Published as a conference paper at ICLR 2026
A SUPPLEMENTARY PSEUDO-CODE
In this section provide detailed pscudo-code for FedGEM algorithm, well detailed pseudo-code for the server computations both in the collaborative training and final aggregation phases.
A. DETAILED FEDGEM PSEUO-CODE
Algorithm FedGEM
Input: Number of communication rounds T, Number of local steps Sg. Final aggregation radius final and fixed kig
weights TR2 VKg [K,], Vy [C]
Output: Final (%. for all components ky [Ky] and clients g [G]. inferred R
1: INITIALIZATION
for clients g = 1.....,C parallel
3: Initialize %% for all kg [K,] via k-means++.
4: end for
5: COLLABORATIVE TRAINING
for round 0.... T do
7: Clients
8: for clients g 1, G in parallel do
9:
B/- (t0-10)) - 0 1) for all ki [K,]
10:
for step $g I...., S, do
Compute The (mn, of (t via E-step in (1) for all kg E [Kg] and samples E [Ng].
11:
Compute Mr, (0) (t 1)) via M-step (2) for all ky [Kg]-
12:
Update (c.*g) May (0 10 for all ky [Kg]. end for
kg
14:
Solve for Eky
(0)
arg max e Juy
(ft)))) via problem in 3 for all ky [Kg].
kg
16:
if then
Transmit tuple (tt.,g)) (1) for all hg E [Kg] to central server.
17:
else
Transmit tuple (t,))))) 20: end
19:
k ssbe for all k [Kg] to central server.
21:
end for
22:
Server
23:
ift T then
Update (0 R() server suaver_ (1.8%, _ ky [K₂] and g e [G] via Algo-
24:
rithm in Appendix
25:
Transmit 0 (++1) clients for all hn [Kg] and and g [G].
kg
26:
else
27:
FINALAGGREGAA Compute (0 R*)
28:
server server_final_ Lerrer_.iaa
( for all h e [K,] and
g E [G] via Algorithm B in Appendix(A.2
29:
Transmit O to clients for all h E [K,] and and E [G].
30:
end if
31:
end for
16

---

Published as a conference paper at ICLR 2026
A.2 DETAILED SERVER COMPUTATIONS PSEUDO-CODE
Algorithm 2 server_update
Input: On, and Ekg for all clients g [G] and components kg [Kg] the 9th client Output: Updated 0%, for all clients
[G] and components hg
[K,] at client g.
R
I:
Initialize K 0.
2: Initialize set Ti, containing only Os, for each [G] and kig E [Kg]
Initialize comp(g, kg) assi gned False all g E [G]. Rig E [K2]
4: Initialize comp kg) supercluster - Null for all E [G]. ky E [K,]
5: for client 91 = .,.., G do
6: for component kg L,..., Ky do client 92 91..... do
for component hig ,..., Kg if MM, Mr (0/) & VEK, / then
10:
D* -M,, + clip 0.5, 5 VERo
(Ml,, (0') - Ma, (0',))
w
11:
Tk., - Tha uu*
12:
Thys Thgz uv*
13:
assignnee if comp(gn, kyn aassigned = False comp km) assigned True then comp(yn, kot) supercluster
(o, supercluster. complgn, Kn ). asssigned - Truc.
15:
16:
else if comp k, = Truc & comp(gz, assignedd = Falsc then comp kg.) superclssser comp(g), kg.) supercluater. comp Kgg assigned True.
17:
19:
else if comp k ass gnedd - True & complg2, kg.) ass. gned - True then
20:
if comp km supercluster != comp -up then
21:
comply kg) supercluster comp kn supercluster Vky such that comp(g h supercluster comp(g2, kzz) supercluster. - -
22:
23:
Reorganize supercluster numbers for all components.
24:
end if
25:
else if comp(g,, kg assi gned False comp koz) assi gned assigned
False
then
26:
K k + 1.
27:
comp k supercluttrr - K.
28:
comp g2, ) superclsserrr - R. comp (g,, Kn asssigned True.
29:
30:
comp (g,, assignee True.
31:
end if end if end for end for
32:
33:
34:
35:
if comp(gl, Kg assignedd False then
36:
-- K comp kg supercluster= R. comp( kgg a.s..sssinned + True. 39: end if
37:
38:
40: end for
41:
end for
42:
client g = 1, 1,... do
43: for component H = I...., K, do
44: 6%,, - aggregate of elements in Tey
45: end for
46: end for
47: K K
17

---

Published as a conference paper at ICLR 2026
Algorithm server_ letrrverrrina__eesee wag.giaagtgggggiii...oo.o (O.,..... (kg
dinal)
Input: Okg and Ek, for all clients [G] and components kg E [K,]] the gºh client
Output: Final 0gy for all clients y E [G] and components kg [Kg]] at client g. R*
1:
Initialize K- 0.
2: Initialize set TE, containing only Ox, for eareachg g [G] and kg E [Kg]
3: Initialize comply, k.). ag).assigned <- False for all [G]. Kig E [Kg].
4: Initialize comp(g, kg). supercluster
Null for all é [G], hy é [Kg].
5: for client 91 ...., G do for component kg = l,... Kg do for client = 91...., do
7:
8:
for component hig2 1, I.... K do
9:
if [M, (0)) Mr ( & VEy y then
10:
if comp(gi, ko ) Lassiggen - Halse & comp(s,, k2) ass gned - True then comp(g, Kan sus.cuuuussee comp km )ts.gg..knassgggng... assigged True.
ansi complo,, kg.) supercluster.
11:
12:
Thi Th. Te Thgz < Tagz u Okp1
14:
15:
else if comp(e1, kig) assigned True comp(g2; k2a) assigned False then
16:
complgg, km2) supercluster +- comp(n, kn)) supercluster.
17:
comp kz2 assigned c- True.
18:
Tag2 - Tag2 Thg: Thy, Thy, Oug.
19:
20:
else if comp(yn, k = True & comp k assigned = True then
if cym k superclssse != comp k2z) supercluster then Temp.1 - Th,,"
22:
23:
Temp.2 + Thya'
24:
The - Thg Temp.1 Wky such that complg' http://kkk..spccclllll.. superclstte
comp(ga, Kiga susercclusse uster.
25:
Th. The Temp,a Vky such that comply kg) supercluster
comp(g Kig supercllusee uster.
26:
supercluster Vky such that comp ) supercluster comp(ga, koz) supercluster.
comp(g kg) supercluster + comp(t,, -oupe
27:
R - ---- Reorganize supercluster numbers for all components.
28:
29:
end if
30:
else if comp kn) assigned False complge, kyz) assigned = False then
31:
K - R 11 comp(g1, N supercluster R.
32:
33:
comp kga supercluster - K.
34:
comp (1, hm assigned True.
35:
comp (g2, kpg assigned - True.
36:
Tha - Thas 02
37:
Tha TRou 0km1
38:
end if
39:
end end for end for
40:
41:
K ). assigned False then k - + comp (g), kig supercluster R. complgn ky l assigned True. 46: end
42:
if compppg
43:
44:
47: end for
48:
end for
49:
for client g 1, I,..,, do
component h = I..... . K, do 51: @finll aggregate of elements in Thg
50:
52: end for
53: end for
54: K K
18

---

Published as a conference paper at ICLR 2026
B SUPPLEMENTARY THEORETICAL RESULTS AND ANALYSIS
B.1 POPULATION CONVERGENCE ANALYSIS FOR FREGEM ALGORITHM
In this section study the convergence behavior of our proposed algorithm in the population set- ting. The population convergence of our algorithm relies on the convergence of the local EM algo- rithm each client y to the likelihood maximizers ff,, for all ky [Kg]- As discussed by Balakr- ishnan et al. (2014). this requires that the local Q2 (g(0g) to satisfy the first-order stability (FOS) condition dehned in [ Indeed, if Assumption 3 holds, then BBlakrishhaan et al. (2014) prove that the
population EM algorithm at client converges to the ground truth parameters 0 kg
for component kg
geometrically as follows:
M (00- 0%
(111) 110
kg ll2
vot
E IB2 (B2
kg i0)),
(8)
where ag is the radius of the contraction region for all components ky located at client g, and Mc, (-) is the population M-step map defined next
MR. (0')) = arg max (x) (z,, 0zz) log( Pk, Pkg (k)))) Wkg [Kg] Ong ER"
Now, consider a local GEM algorithm whose update during each iteration is any my, (0(--1)) (t)
(M,, (o!-")) (1)) where the radius v Veg, is obtained by solving the problem in 33. We
show in Theorem 5 next that this algorithm exhibits very similar convergence behavior to that shown in (
Theorem 5 (Local Convergence of Population GEM). Suppose Assumptions through 5 hold.
Consider a GEM algorithm whose iterate m,, (0f at iteration is such that ma, (0
(2,,,,, (0 );: v ( where the radius v Vey, K,
is obtained by solving the population counterpart
of the problem in (3. Then, this algorithm converges to the ground truth parameters 0%, as follows:
[|mm, (0) [t [ 1) 0% 112 <
Bg (t- p01lo(t-1)
0% 112 + c(t)
vol--1) E B2 :0))
(9)
Xg
Kg
where <(t) = 1|M (0(1-1)) 0 ((-1)
12 0 as 1 , 00.
Rg
Proof.
m|m, (0(- 0% 1l| l (0 (((-11)) Ms, (Q((-1)) + Mc,(0
0 kg
01,112
(10a)
||M (((-1))
0% 1|2 ||M (0!t (0(t-1)) -mk, (0 (0(t-1) (0(
(10b)
B1 110 (t-1)
0%, ||2 + || (0(--1)) -m,, (0(:-1))) 1|1
(10c)
Xg
kv
39 110 0t, 2la + ||M (0 (((--1) k ((22))
112
(10d)
Ag ke
where (Tbb) follows from the triangle inequality, 10ch relies on the convergence of MR, (0g) with 8.
1, and (Odd) follows from the definition of mæ, (0g). Now, we consider the
|| (0 g )) of-"1112 term. Firstly. observe that o( for all e [T] are iterates of GEM algorithm. Moreover, recall that we assume that the true log-likelihood of our problem is bounded from above, and the expected complete-data log-likelihood Q, (0, (0')) is continuous in both its conditioning and input arguments. Therefore, by Theorem must converge to stationary value of the true log-likelihood. This suggests that the quantity -
(Wu) 1983), the iterates
0 as f - xc. Finally, by the assumed strong concavity of the expected complete-deee data log likelihood function everywhere, we have that
Q, (0 1000-11)10000110 - (0000 X₂ 1|, (((-1)
2
proving that |M (M,,(0((-1)) (0( 0 o 0 as oo. This concludes the proof.
19

---

Published as a conference paper at ICLR 2026
B.2 SOLUTION ALGORITHM FOR CLIENT RADIUS PROBLEM REFORMULATION FOR ISOTROPIC GMMs
In this section, we introduce the low-complexity Algorithm which can be used to solve the refor- mulated client radius problem in (7). Subsequently, we introduce Proposition the worst-case time complexity of Algorithm
2
which establishes
Algorithm 4 Radius Problem J4, (0') ( Solution Algorithm
Input: Øz,, Mr, ( 0, E.... = Me, (0') 022 ll
Number of iterations I Output: ckg
Parameters:
1: for i O..... do
2:
3: Solve for (() minimizer of Fi (i kkg
big
g(0)
4:
(1) = then
(1+1)
5:
kk &
(.+1) (6)
6:
Eky.th
k. uth
7:
else
8:
kkg.uh
(++1)
2(6)
9:
((+1)
kg lb
end if
11:
end for
where Frr00' ((
the optimization problem shown next.
200)
min Big Meg
Lk,
N Σ
s.t. a.
Ckg 0kK +
The (āng 0²r) (112m, - -Ma, (0,)1
|| - -6% -0%,11 113 -Ex,) akk
)akg"
12 -1
N
Ng
tk +
Σ
The (Zng: 0²)
Σ
Vkg (Zng 02)1 01, 1 00
11
N₂
Oiky 2
Tkg
Bnng 02),
Lx, 3 0.
11g
(11)
Proposition 2 (Local Radius Algorithm Convergence). The Algorithm 4 converges to an optimal solution sx, of the optimization problem in 7 at a linear rate, with a worst-case time complexity of (log((())) per iteration, where Gol is the solution tolerance of the feasibility problem (11)
Proof. Consider the optimization problem in Firstly, observe that the first constraint is non- decreasing in Ek, for a fixed Oxg,' This is because EK. (a?, - ax, Σn, 7h, (Zn, 0')) 0 due the constraint that ork, - The (Zn,, 0y). Moreover, note that by the strong concavity of (,,0))))) we have that Ek, must obey & VeR, & [|Me,(0') follows from the strong concavity of (0,10') that Ek, must always be feasible solution to the problem given that the algorithm has not yet converged. This is because the strong concavity of Qk, (0k,,10 for the GMM discussed in Section suggests that I
0%1,1z Furthermore, it also
7h, (Zn, n 0 l 0%,, 1113 E 7h, (Zi,, , (,,,,,),) ( Mr, (0') (0)) with equality attained if and only if 0'
20

---

Published as a conference paper at ICLR 2026
Mr, (0'). Therefore, ak2 be made arbitrarily large make sure the constraint is satisfied. Combined with the uniqueness result presented in Proposition]] [ these facts suggest that we can use bisection approach such as the one shown in Algorithm t obtain the optimal radius. Moreover, this also suggests that we can use the optimization problem presented in (T1 to check the feasibility of a given Ex,- More specifically, an optimal 122 0 suggests that the constraints already satisfied, and therefore the estimated Éx, is feasible, and vice versa.
Now, observe that the feasibility check problem in is a quadratically constrained quadratic pro- gram (QCQP) with 3 constraints and 2 1-dimensional decision variables. Therefore, it can readily be solved via the barrier method with worst-case time complexity of (log(e 1), where Gol the solution tolerance of the feasibility problem (Nesterov & Nemirovskij 1994).
B.3 SUPPLEMENTARY THEOREMS VERIFYING ASSUMPTIONS FOR ISOTROPIC GMMS
In this section, we verify three key assumptions to guarantee the convergence of our proposed FedGEM for the GMM discussed in Section 5 More specifically, we begin by proving that the population Qg (0,0'''') function associated with the GMM under study obeys the FOS condition in derive the radius of the region for which the population M-step map for this model is indeed contractive in Theorem 7 Finally, we derive the upper bound on the distance between the population and finite-sample M-step maps in Theorem 8
Theorem 6 Subsequently,
Theorem 6 (GMM First-Order Stability). Suppose Rmin 2( min(din(,,)) Then the func- tion Qg (g(0g)))) associated with the GMM described in this section obeys the first-order sta- bility condition defined in I for all Ok, E [B2(0% ky 0g) Vkg E [Kg]. where ag & R
2
v /min(dd Kg} max|4 2[log( (t///4444....), 8v33. That is [VQ,( - VQ(M 122 & Bo -0
with
Bg = (1+ (1+ 2/) + Ty max.
where Tyy is constant depending Kg. Rmino Og. d, Tiing and Taax whose explicit form can be found in the proof, and
(- (-(Ez) Rymin 2
2
By = K22K 4)(2Rmax + minfd, min(d,K,)))
K21) exp
exp
0g
/minfdd d, K21/8
Proof. We begin this proof by studying the FOS condition for each component k, separately. Firstly, note that
Ver, Q(0g10,)) IT,, (x. (e.0,)(z 0%,,11
Therefore, plug in Ok, Mr, (0g) obtain the following.
V Mr, (0.) QMM 0,)) V Mc, (0,) QM, (0,, (0,)
(12a)
E [(Tx, (xF 0g) Th, (x, 0 0*) MR, (O,))ll2
(12b)
E [1%,, (x. 0g) nk xx, )))(= x 0x, + 0k, Mr (0,))1l1z
(12c)
Ex [17k, (x. 0g) Yk, (x, 8t )) 0x,))
Em [(,,, (xx 0,) - Yk, (x. (Mr, - 0x,))lle
(12d)
Em [[k,, (x. 0,)
The (x, 0 )) x 0x,) 1112 +
A
|a [ (2,, 0g) - Th, (x, (*))(((aa.).)))) (0g) 0k )]1k
(12e)
where (12g) follows by the linearity of the expectation operator, and (2c) follows from the triangle incquality. Now, is established in Theorem in (Yan et al. 2017) that
3'
K |00k
A
0 1|,,
(13)
K
21

---

Published as a conference paper at ICLR 2026
where By is defined in our theorem statement. Therefore, it remains to obtain upper bound for Ag. We begin this as follows.
2
| (1kk, (z, 0g) 7he (2r, (0)) (MR, (0g) 0k )]l2
(14a)
|| (7*. (xe, 0g)
-k, (x, 0 (Mss (0g) 0z.) l|
(14b)
Lo l7ke (x: 0n]] Ex lTke (x. 00 Mx, (0,) 02, 112
(14c)
Ex Lvk, (x. 0,)
EBze (ite (x, 0g)] Ex Tik (a. 07)
kkg
(14d)
Exe 1k. 0g)
Ean [Tke, (az, 0g)] Te 7kk (ar, 0]]
Ege lik, (x, 0g)
0k 1ll
(14e)
Ex 7kk (x, 0y]]
Ans
A2o
where (14b) is obtained by realizing that Mr, (g)) is comprised of expectations in and thus is no longer random, and (I4c) follows from the linearity of the expectation, and from the fact that the expectation terms become scalar quantities, and can therefore be taken outside the norm. Now, must obtain upper bounds for the terms A2. and A2a- bounding A2a. we proceed by obtaining an upper bound for the numerator and lower bound for the denominator. To achieve this, firstly observe the following.
Thy Oxx
K2 Σ 122
En Iik, (x, 07]1
Tm, pl (xxx0
dx
(15a)
K i J o(x10t,))
x
Tk ((x) 0 )d
(15b)
Tky""
(15c)
Next, we examine the Ex IT,, (x, 0,)l term follows.
K₂
Ex lik, (x, 0y)]
Ti Ex likk, (x, 02) .N Ia)]
(16a)
K
Ti
(z)
(16b)
J
which follows from the law of total expectation. Now, we analyze two cases as follows.
Case 1: lg kig In this case, we have that
[ Tke (x. 0y) ((x) dd < S 6(x10f ((x=10%)dd = 1.
Thus, nI, Sm Th (ar, ()) 0x da = Th,
Case 2: ly kig In this case, we have that
Th, exp( -11 0k,118 112)
mkg (x. 0,)
(17a)
Ki أ. 1 Ti exp(-xpp - 0,,113)
Th, exp( | 0k (l))
(17b)
11 exp( 2 01, 11,113)
Wh exp
(1|æ 2
0x, 11 || 0,,113
(17c)
22

---

Published as a conference paper at ICLR 2026
Now, define event Akg.r fo: some constant, which will obtain bounds for later. Since Lg * lig we can obtain lower bound for the quantity | 0₁ 2|2 for X E Ar,.,. via the triangle inequality as follows.
Ia),
07,112 -0 E r'g), where 79 R is
2g 112
|| rr 0%
0k ll2
, 0k2 1l2 1|0
2|12
1
0% 0% 0g, 112 rg
1 0% kg lla 1
0% l12 - Tg
Rmin 0g Ts-
Similarly, can obtain upper bound for the quantity || 0kgll2 for ARgir as follows.
|| - 01,112 || 07, + 0%, - -1,112
|ac 0ī, lla + 1 01,11
rg ag-
Therefore, we have that
Tha exp
(| 0
1 | 01, 11)
Th T
exp
1
RR
2m (ag r3))
TI
with the requirement that rg Ruin ag ensure the negativity of the term inside the exponent. This allows us to write the following for l * kg-
S Nhe (x, 0g)6(x(()))) dx
The
R2
22Rmin (ag ra)) 6(x100, ddx+
exp
Akginyy
Tig
[
The (x, (x, 0g) 6(x) ((x)00 da
(18a)
E1 pxx000 dd
The exp
RRiin
2Rai (ag + rg))
+
(18b)
Ti
Tkg
exp
(Rain 21min + rg))
P([1 0f, lla ryle NOO0, La))
ni
(18c)
Th
exp
RRain
2R (ag + rg)) + exp
rg Vg
(18d)
where (Id) follows form standard tail analysis shown Lemma 8 in Yan et al. 2017) for
Tg > 2v
Putting together the two cases analyzed previously, we obtain the following.
Em [Tk, (xx, 0g)1 & Thy
Σ
Aly
The TI,
exp
1 RR 2R (ag + rr))) + exxp
ry Vg
1 R
rgVd
Tkg Tk, (Kg 1) cxp
2 (ag + +rg)) + (1 Tkk, exp
This allows to directly observe that
.ryvd
Ex L7x, (xc 0,)1 Ex (Tx, (x, 0%) & The (K,-10 exp
- (R2 2. (a, + rg)) +(1 +11-mx,)))) exp
2
23

---

Published as a conference paper at ICLR 2026
Now, it remains to obtain a lower bound for Egl",,
Rt, (z, 0g)). which we do as follows. First, define
an event BEg.r fx: a Ni Ta): || o lla rg]], where rg is such that 2 &
rg Ruin ag
previously. Then we have that
Em Tk, (x, 0,)] ,( [Tk, (x, BB)
rnJ Pr LTk, (xx, ())
(19a)
P(B Eu 7k, (z. 0g) Brgg,
(19b)
where 19a) follows from the law of total expectation, and (19b) follows from the fact that Th, (ar, 0g) is uniformly lower bounded by O. Now, consider æ E BR For in E [Ka], ig * kig we can lower bound the quantity ||
0,gll2 via the triangle inequality as follows.
|| 021l22
0 kg
+ 0% ij ll2
|10,
0% ll2
| 0 kg ll2
||0}, 0, 0 0% lla rg
1l2 Tg | 0 la |10 Jg
Rmin
rg ag
Similarly, for ig = kg we can upper bound the quantity follows.
| 0j,12 via the triangle inequality as
||
02 1l2
|| 0
0% 0k, 12,112
< x t ll2 + 110t 0k,1 1|,
rg + ag.
Now, let write Yk, (x, 0g) differently to simplify the analysis. To do that, let define the follow- ing.
1
3x 0g) =
(ax, 0g) Nk (xx
Σソ J
The ( 0 kk)))
K
T exp
- (1|20
1
02,112 -1100 - 0kk 02 112)
2
3g
Therefore, given that B write
7h (x. 0,)
(x, 0g)
Fia exp [- ( -1 The
0j,1 - || 0x [2)
Tie exp [- 1/2 (R2 min 2 2Rmin (r, + ) 1 + Σj, ElK #ko th, Tkg
1- 1- exp
1 [R
2R (rg + ag))l
This lower bound on "ik (xx 9,) in the case where x Beg allows us to write the following.
Ex Em (x. 0,), B.grol
1 1
I-Kka
exp
[ [
2R (r, + ag))]
Whg
24

---

Published as a conference paper at ICLR 2026
Then, bound Ex Tkk (xx, 0g) as follows.
P(Bk,,:u))
En S/k (x, 0g)1
(20)
1 exp [- 2 R2 22 (rgg ag))]
1 1+ xkg
P(æ: N Ny Ta)) P([ || 0r, 112 rgg NO Ia))
(21)
1 1-Tha exp | 2 [ R (rg + ag))]
#kg
R P( 0% N2,ll .21l12 Tglar NN Ia))
(22)
1 The exp [- { R R (r, + 0g) (g))]]
1 +
*by
Tke
- P([1= 0 kg |l2 2 rgl ~ N k. Ia)))
(23)
1- exp [ (R - 22R (r, + 0g))
-royaa
kh
exp
(24)
+
n2g
exp
1/2 (1e
-2R (rg + ag))]
8ky
(25)
As result, our previous analysis allows to write the following uniform bound for the term A2a
for all ky e [Ky]-
Em [,,, (x, 0,)] E. [Tx,, (ax, ]
7hg
E 1x,, (xx, 0,)l
maa (K, - 1) exp | 2
(R min
- 2R (ag + +r)))] (1 Tmin,) exp
(- Lava
2
& max
ru 2
Kaing
1 1-exp
1-
Fuity exp
[ (R2 -28mm (F. +ag))
1-oxp
Tmax
1
exp
1/ (R
Ruis (1' +o,1)
Fuing
Wnitg
1
14
L-aging exp
[ [R2
2 (r. +a,))
25

---

Published as a conference paper at ICLR 2026
Thus, it remains to obtain upper bound for Azs. We achieve this as follows.
Azs |B (Ts, (xe, (z) - 0s,)]ll2
(26a)
= [|BBr[Ti,, (x, 0z) 0n )] Em [Tx,, (xx, (2) - 0c,)] + Eal (x,, (xe, 0z)
ON,, 1ll2
(26b)
<
|| (T (x, 0g) - 7h (a, 0g) x - 0k.) [ss) + || TTk, (x, 0g) gx) 0x,)]ll2
(26c)
9
|10,,
0% 1l2 E E (ar, 07)
Eal Eik (2, 0k,00,,,l 0k,
(26d)
Kg
3y K
Tky ((x)
|10
0 112
Σ
mn Ql x (
dæ
gg
K
T o 10,
jo
my=1
Th o
Σ
Kmg 5z 0mm))
dx
ik 11 JJ 612)
(26e)
K 9
|10, 0% 1l2 + Tkg 10x,, - 0 ll2
(26f)
Kg
9/
Kg
03 1l2 + Tmax, |10,
Tmax,110x, 11xx 0% 9%,1
(26g)
K 9
where (26c) follows from the linearity of the expectation and the triangle inequality, and (2dd follows by leveraging the upper bound derived in Theorem in (Yan et al. (017) discussed earlier. Therefore, we can summarize the results we have obtained thus far as follows.
V Ma, (0,) (0)((
VAL, (0,) ((,
3/
K, Σ
3
K
g
|10,
0 112 + "
9
| jo
0 lll + Taxxy 10x, - 0%, ll
K
K
ja
jo
(1 + 3 Br
K
|00 0 1l2 + Tig Tmax, | 0% 112-
Kg
Now, observe that
(Q0(M (Q(M)(0,))0))
K.
VM., (0,)) (,,,(,,)))))
V V., (0,) (MM,
(1 + T )
Kg
Σ
|00;,
0+ ||2 Rr max 100,
0% ||2
K2
26

---

Published as a conference paper at ICLR 2026
Expanding each term inside the square root results in
[VQQ(M,(0g)(.)) (O(M,(0,))0
(27a)
Kg
(1 + 3
Σ 100
0% 1l2
K7 9
(1 + Te) 1/
K₂
9 Tmax, 10x,
110 - 07,, 21 + x² mmax, 110t,
0 11
Kg
hg
30
(27b)
2
(1 + 2
Kg
110,,
07 ll2
+
Kg
(1 + xzg 3/
K2
K
Kg Σ
2
it
T m
10x
0%, 112
Σ
100,
0 112 + x muxx
100,, 0% 1
Kg
Rig
3u =1
Rig =1
(27c)
(1 xg)2
K
Σ
110 - 07 1l +
Kg
2
(1 + )1 IT mmax
Σ
110;,
00, ll2
x Mmx 110,
(27d)
K
Ja=1
(1 + 23 2(1 + n²y) 5/ T T T
maax
110
0-
(27e)
To see how we obtain (2d consider vector u of length Kg. all of whose entries are 1, and a vector e of length Kg. with 1 entry |00k, 0% lll. Now we can rely on the Cauchy- -chhwarz inequality to write:
g
K u 0
Σ
100, - 07 122
$ ||x|| ||u||2||u|| |2||u|122
K
KK,,
0 07,
/ 1g
01112
Thus, this allows us to see that
2
K
Σ
100,
0 |12
< K2 -0,13.
Therefore, we obtain our final result by taking the square root of both sides, resulting in the follow- ing:
VQ(M,(0,)10.) -VQ(M,(0))0 < ((1 ((++T'))5
T'') T maxx,) 110, 0*
3,
27

---

Published as a conference paper at ICLR 2026
Theorem 7 (GMM M-Step Contractive Behavior). Suppose all the conditions of Theorem o hold, Rha 16
the clusters are well-separated (i.e. Rmin >> 1), the data dimensionality < is sufficiently large, and the radius 0g of the contraction region client is such that
Ramin 2
Ki Naiig Cog Taong
0g Ca.
vdo
log
max
Rmax, min Kg}
1+crg
for a sufficiently small constant 0 <
Ca.
and 0 Cx.
Wamty defined in the proof. Then, the
contraction parameter of the population M-step of the multi-component isotropic GMM is
2x < 1.
Ag
Proof. In order guarantee that the population M-step is contractive for the GMM described, must show that
Mmin
T max
83
(28)
Ty
We can plug Sy from the statement of Theorem into inequality 28 and rearrange terms to obtain the following.
Rmin
2v2
K3(2kg 4) min(d, Kg))²
0g <
log
2
//mind,
gg
Faing Rang
1+
Subsequently, we can combine this upper bound on with the one presented in the statement of Theorem to obtain the following.
Ryin
2V2 mmn(d d, Kg}
K3(2ng + 42 + min(d, KgJ)²
0g &
max
log
Mon n Amaz
1+
min(d, Ky} max 4 (oog( Rmin/ 4)] 8v3)
Now, we derive an upper bound to the maximization term. In doing so, we begin by obtaining upper bounds for A₁ and Ag as follows, considering A first.
2
K2(2kg + 4m + min(d, K
A1 =
log
/minfal (d, Kg}
Faing
I+
K2 + 4)(2 max + minfd. K 12
$ log
(29a)
Tain
Tuang
KRtigg (mxx min (d, Kz
E log
(29b)
Amiz, - Kmany
I+
C1 KgKKgg
log
+ log (2 + min(d, Kg})
(29c)
Maing Tussa
1+
K3 Tam, #. Teasy +++
< Sc/minfd Kg)
log
C2 log (CyRaus + e min(d, Kg)),
(29d)
28

---

Published as a conference paper at ICLR 2026
where (29a) follows by plugging in constant c, and recalling that minfd, Kg) (2bb)
obtained by noting that Ki > and choosing e > 6, and (29d) again uses the fact that min(d, Kg) > 1, well as the monotonicity of the log function, and plugging in constants C2, 03 1.
Now, derive upper bound A2 follows.
min(d, Kg) max (ming((mmn/4))]1 : A2
minfd, Kg) maxfcr / [log( Rmin/ /4) 4)]+
(30a)
minfd, Kg) maxfcr log( Rmax + te, C2)
(30b)
minfd, Kg) max Alog( Ranax + te) C2 Vooooo max + e)}
(30c)
min(d, K (c /og Rmar + e) eoy/ Rmax e))
(30d)
d' v min(d, (ggndd K log( Ranxx + te).
(30e)
where we obtain (30a) by rewriting the constants C1, C2 > 1, (30b) is obtained by noting that
Rmuxx 3 Renn Roi and adding an term inside the log to ensure that it is greater than or equal to 4
This allows to obtain (30c), which is then upper bounded in (30d) by noting that both of the greater than O. Finally, obtain the final result by combining
terms inside the maximization the constants C1 e2 into new constant c.
Given the bounds derived for A₁ and A2, we can write the following
max max A2)
C2 K2 K2 T. Faui
V min(d, Kg} max
log
+ log (caRmax + +e minfd, K2)),
Fiing
1+5
Vogl Peaxx +
(3la)
Ci KYKgg
< V min(d, cd) log
+ c2 log (CaRmax + + min (d, Kg))
(31b)
Maiiy TT. Taarg 1+
Ci K3KK
$ min(d, min(d, kg)(c
log
+ C2 log (Ca Renx + e) + C2 log (min(d, K2))
Taing T Tman
1+
(31c)
C3 C3 Rmux + e, min Ky) })
K2 K3
& min(d, (g](c+ d'
Cz log
max
(31d)
Knin, x Tanry 1+50
K3
V min(d, Kg)0
log
max
Rmax min Kg
(31c)
Tuis n² Kman
1+
where (31b) follows by absorbing A₂ into A1, 31c is obtained by noting that each of the three terms within log functions are greater than 1, and therefore the log their sum is upper bounded by the sum of their logs. Finally, (31e) is obtained by eliminating all constants. Therefore, obtain
29

---

Published as a conference paper at ICLR 2026
the following condition on dg-
Rmin - Vmminfd K₂)
K K2rgg Fornn x Rmaz
0g 2
log
max
Raxx min (d, Kg)
1+
Now, recall that Ti (defined in the proof of Theorem 6 depends ag. However, we note that under the conditions of this theorem (i.e. Raii 1, sufficiently large d Ram
T/J approaches 0. This is true for any 0g < Co2 A dvd) for some sufficiently small constant < Co, < 1 as long as
Tg (from the proof of Theorem 6 is set to its lower bound of 2 Therefore, rewrite our upper bound to ensure the condition on og is satisfied as follows
Reinn 2v
Kigg
0g & Cag
log
max
Rmax min Kgg
2
Tiity -Ceg Tomg
1+0eg
where 0 Cxg Knnzy
Raing is constant equivalent to the value of ni evaluated â, Ca, Eign - 2 d)-
We note that there must exist Co, such that the condition on Cr, is satisfied. To see this, note that Tainy holds trivially. This is because, mentioned previously, Tig is close to O under the conditions of this theorem due to the exponential terms. Additionally, we observe that Ti is monotonic and continuous in ag. and therefore is also monotonie and continuous in ca.- Thus, there must exist some Ca, > 0 such that Cr2 of T.g. This concludes the proof.
with Cag 0, Cng
Fannty
Taitg
Finally, Cx, > holds trivially by definition
Kmang
Theorem 8 (GMM Finite-Sample and Population M-Step Distance). Suppose Let ((Ng)
all the conditions and definitions of Theorems 6 and δ ( (maxx K11 ( R max(1, log(kg)), Romx))
7 hold.
d /Na }
Moreover, let MR, (0,)
and Mr, (0g) denote the population and finite-sample M-step maps associated with the GMM described in this section, respectively. Then, we have that
Mr, (0g) Mr, (0g) 11,
1
(D(Ng) + 22g
2
sup 0gchg
& PN
log
2N
1g
with probability at least (1 - exp (oodlog N₂)) (1 ngy, where e is some positive constant, Ag =
1 O2 ag), and
royd
Twin
- exp
1 log
2
FN₂
Taing
exp
[ R2
2R (T, ag))]
2.N,
Proof. Recall that the finite-same sample expected complete-date log likelihood function associated with out GMM model can be expressed as follows.
N
Kg
200000))) =
Σ
Tog (Zng : (log Th, + log (0g)))
N
n1
Moreover, its gradient and Hessian can be written follows.
VoN, (,,00,)))
1
Thi (Gng (2n, Ok.).
(32a)
Ng
1
2,100)
Tky (Z 0',
(32b)
0kg
Ng
30

---

Published as a conference paper at ICLR 2026
Now, observe that the Hessian is a diagonal matrix whose eigenvalues all equal to the empirical expectation of the responsibility function 7k, (#n, o²)_ Therefore, the function is strongly concave, with strong concavity parameter ZNy, which we define explicitly later.
Thus, by the strong concavity of the (0(0,)0))) function we can write the following.
-Ma 0,)l
MR. (0,)
2g (MR, (0,))0,) (M,,,, (0,))0,)) + Veng (0))),)00)10,)) (Ms, (0,) - -Ma, (0,))
(33a)
(M,, (0)) (M,, (0g)
(33b)
Similarly, we can use the strong concavity to write the following.
FN₂
MR, (0g) Mr. 0,)1
< 2g (0,))0,) 2, (0,) + Veng (0, (MMx (0,) - Mx,(0g))
(34)
Summing up inequalities
33b
and 34
we can obtain the following.
pN.
Mc. (0,) Mr, (0.)
2 (0,)),,) (Mx, (0,) - (M,,(0,))
(35a)
V Von (.(M,,(0))) ,)
Mr, (0g) Mr, (0,)
(35b)
Von.
Von, 20 (Mr, (0g)
Mr, (0,) Mr (0,)
(35c)
where (35b) obtained via the Cauchy-Schwarz inequality, and (35c follows from the fact that V ohn (M (0,))0)) 0 as it is evaluated at its maximizer.
Now note that if M (1) (00) M (k) (0').. then the finite sample EM algorithm converges trivially by the convergence of the population EM algorithm. Thus, we focus on the cases where M())( o
M (k) x (0').. this case, we can write the inequality in (35c) as follows.
,N,
Mr, (0g) Mr (
V Bi. 2 ( 0,)) Vo., Og (0,)
(M,,
(36a)
Vo., Q, 02)) Vo. Q, (Ms, (0) Os.))
Vot, (0,0, Vo, 0,, (Mr, (0) 0m.
(36b)
V 0 (,000, V e., Q
+1 O, 0, (0) Ox,)) Vo., (x(000)(MM,(00) - 0m0)1| (36c)
V o (0,00
V Vo.. (0
+
7kg ann 0y)) - Ex ITKk (x, 0y)l
|M, (0,) - -OHalla
(36d)
Ng ny
N;
V or 2,(0,10,) - -oe.. 20(0,10g) +
Σ
7k, (Z 0,)) - Ex Iyke (xx, 0y))
ag.
N ng
(36e)
31

---

Published as a conference paper at ICLR 2026
where (36b) follows by writing the Taylor series expansion of the function f: RR - R f(0,) V o2, ( gg gg), (36c) follows from the triangle inequality, and (36d) uses the fact that the Hes- sian of both the population and finite sample functions diagonal with identical entries. Finally, 36c) leverages the fact that MR, (0), 0x2 E B2(6k, kg ag). Now, suppose we have training set X comprised of Ng IID samples. Then we can write the following.
N₂
N₂
P
Σ 11y
Yk
1 ng. 0g))
E
Σ
Tko
(ā 0g))
< Nog5
N2
N2
Nhe (ing 0,))
Σ
Ex
[ite, (x, 0,))]
< No
11g
n1
N₂
Ykg (ing 0,))
Ne [iky (2,, 0y]]]
$ No
N₂
P
Ng
Ykg (In. 0g)) Ex 1ik, (ax, 0,))]]
< Nooo
Ng nly
N₂
1
P
Tke 0g)) 1 Em ITNe (x,, 0g))]
Sa
Ng
m1=1
for some small Ey e IR.. Moreover, by Hoeffding's inequality have that
N₂ P
N
Σ
Th 0g))
0n))
E
Σ
The (ang 0n))
> -1-2 1-2exp (-2N < NooS
Therefore, this allows us to write the following.
1
P
Pag (.., 0,)) Ex [7k, (xx 0,1]]
<Cg
> - 2exp 1-2pexp
(-2Ng52)
Ng
Now suppose set 77
2 cxp (-2Ng then obtain that with probability least f7
Σ
Th (Zn,, 0,)) Exe [Tk, (x, 0g))l <
log 2.N,
2 17
(37)
Ng n
Now, let us define by as the contraction region IT (2 ky. ag). Anned with the previous prob-
abilistic result, we can say that with probability least exp (-od log N₂)) (1 n1).
sup 0eh
Mr, (0,)
Mr (00)
& 1 FN,
Suuns (N + 2ug 2N
log
2 flg
(38)
where Wans/( (anss(N,) is defined in the theorem statement, and (38 follows directly Theorem in Yan al.] 2017) which states that for problem setting, (uni/(N,) with probability at least exp (-ed log N₂), where e is some positive constant. Finally, note that under inequality we can say that with probability ng-
from
suPeee
Ve., 2,
Vo., ( 00)
B7
N
Tky (mn, 0g)) E. Ti, (x, 0,))] V/ 2.N
log fgg
N
m==1
(- rova
Tiin,
exp
2 log 2N,
FN₂
Kaing
exp
R2
2 2 (Tg + [g)}]
17
Aucc
32

---

Published as a conference paper at ICLR 2026
which utilizes the lower bound obtain Eg Iik, (z. 0g)] in the proof of Theorem 6 This concludes the proof.
B.4 PRELIMINARY DIFFERENTIAL PRIVACY DISCUSSION FOR ISOTROPIC GMMS
We study a potential method for the clients to enhance the privacy of their data. More specifically, we consider the use of differential privacy (DP). In doing so, client g adds Gaussian noise to the
estimated centroid (()) of local cluster h at global iteration t. Next, we define various fundamental
Kg concepts in DP, first introduced in Dwork & Roth (2014).
Definition 2 (Differential Privacy). randomized algorithm C said to be (p, jiffe--lifff
private if for all S Range(C) and any two neighboring datasets X and X' of the same size but differing only in one sample we have that P (C(X)) E S) & exp(p) (C (C(X') E S) + μ.
Definition 3 (fz-Sensitivity). The fz-sensitivity A2())
) of function f X TD is defined as
22( maxx.xx
(X) - f(X') 1l2, where X and X' are datasets of the same size, differing
only in one sample.
Definition (DP via Gaussian Noise). Given a function f. X - R with lz-sensitivity A2(s), then C(X) f(X) .N(00 is said be (0, (ij]-dffreentilllyy private if >
Sa(f) 2 loo( L.20
Now, consider the map (0') RRKK, - RdK,, which maps the current vectorized centroid estimates 0g at client g to the vectorized maximizer associated with all clusters. In order to guarantee differential privacy, each client g independently perturbs its local (O')') before sharing it with the server. other words, client g shares a perturbed (0') = (O)) + NO, ((,,o"La,)), Next, we present assumption on the support of the data, followed by Theorem p where we establish the required standard deviation of the Gaussian noise to ensure that this map is differentially private.
Assumption (Bounded Support). Any sample Rn, at client g E [G] such that || llz & B R.
Remark Note that Assumption / is not restrictive in practice. This is because data is often col- lected via sensors other methods with known ranges, and can therefore be readily normalized.
Theorem (Client-to-Serve Communication DP). The perturbed estimate My(00)) = M, + ((0, TTx,)) sent by client g to the server at global iteration t of the algorithm is guaranteed to be (p, didfferrrnti private if the standard deviation of the noise satisfies
K
3B B
2B R.
2
log( 1.25
P
where Bu is defined in Assumption 7 and Byg is constant depending on client-level parameters such as ag. Tming: and Tmax, and is explained in the proof.
Proof. In order to establish the differential privacy of the map (,,))) via Gaussian noise, suf- fices to derive its fz-sensitivity A2 (2(M)((0)) We begin by considering the maximizer Mr, (0') associated with cluster kg. Now, consider the two datasets X, and X' g: both of which have N₂ samples. Without loss of generality, suppose the datasets differ only in their last sample. That is, the first N,, - samples are identical across the two datasets, whereas the N7th samples in X, and 2", are a* and a', respectively, with a* a' To simplify notation and to highlight the datasets as the input of interest in the mapping function, let us define the following.
y
N₂-1
N
8kg
Yh. (n,, an,
Wky =
Yk, (Zn, 0²))
n1=1
7 Tk, (xt* 0²),
7' 7k, (5) 0²),,
(39)
33

---

Published as a conference paper at ICLR 2026
fx(X) =
(.))
fkg(2) == -M, (8)))
Moreover, assume without loss of generality that 7 > y. Now, we analyze the lz-sensitivity of Jag (2) as follows.
Ik5k, (X) [kg 122
8k, + a=
3Rg + 7/ '/x
(40)
Wk, + g wko 7/ 12
(wk, + (8R,, + x+ - (wi, + 7 (88, + +
(41)
(wi,, + (w) + 72)
2
(wk, + (WKK +-7 [1 + (BK, + +1*x0) (wk, + 7 (Bk, + +
(42)
1
11(2' - --0) )8k, + wk, 70 x² why //' + + (x=" -2')ll2
(43)
(wk, + + 1
[118k, 1|2 + wxy, |10 + wk, | + 11 - a 31112]
(44)
(wk, + イイサト2
No-1
1
Σ
Wkg (Bn: 81)116mm 1l2 + WK, 1| ll wk, || ll2 + 1|
6'112
(45)
[wk, + 7
=1
(3(wk, + 77)) B2
(46)
(uTK, 7:
3B
2B
+
(47)
Bi 13
Ta
where 40 follows from the definition of Mr, (0²)), 43 follows from the assumption that 7 > 7* 44 follows from the fact that 0 & 7*
and 0 & sum is upper bounded by the sum of the norms of the individual terms, and finally, follows from Assumption] 7 and the fact that y* is positive. Note that we denote the lower bound (Wk, +?") by B, which depends only on parameters specific to client g but not to cluster kg. This lower bound can be taken as ēN2, which derived in the proof of Theorem g Now that we have the fz-seniitivity
1. 45 follows from the fact that the norm of
of the maximizer of single cluster, we obtain the overall fz-sensitivity of the map over all clusters as follows.
K₂ Σ [1 (X) - Sky (2')
(x) - 13(22')12
h
3B
2
g
Byu
+
B²
3Bn
2B
Kg
Br
B2
This concludes the proof.
Remark 4. Note that we readily obtain upper bound the distance between the popula-
tion Mr, (0," (t) and the perturbed, finite-sample Mr, (0, (1)). Therefore, by the same argument in
Theorem 2 we can argue that iterates Mr, 9 (0, (+) converge the neighborhood of the ground truth parameters 0% However, the iterates do necessarily converge to single point within the neigh- borhood due to the randomness of the added noise. Further work should explore this convergence behavior more carefully, as well as the implications of DP on the privacy and convergence of the uncertainty radius computations.
34

---

Published as a conference paper at ICLR 2026
B.5 COMPUTATIONAL EFFICIENCY AND COMMUNICATION COSTS
B.5.1 IMPROVING THE EFFICIENCY OF SERVER COMPUTATIONS
While the pairwise server computations described in the work are intuitive, they can be inefficient for large-scale problems. To see this, consider worst case where each client y has local number of clusters Kg O(K). In this setting, the server would need to perform roughly O(G operations, which can be very expensive for very large G or K. To improve efficiency, the server may leverage a d-dimensional binary search tree (commonly known as KD tree) (Bentley 1975). More specifically, the server would store the estimated cluster centroids Ma,, (0(- for all clusters kg [Kg] shared by clients g [G] in the tree. Subsequently, the server would iterate over each centroid Mr, (0(t=1)), obtain obbainn its M nearest neighbors (by slight abuse of notation), then check for uncertainty set overlaps and perform aggregation described in Section 4 The construction of the tree incurs a cost of ( log GK) (Friedman et al. 1977), whereas a single nearest neighbor search incurs expected cost close to Olog GK) (Friedman et al. 1977) in practice. Therefore, assuming that the number of overlaps between uncertainty sets is significantly smaller than the total number of available uncertainty sets (that is M < GK), then have that the total cost of constructing and using the binary search tree would be close to (2GK log GK) OGG log GK) in practice. This approach can improve the efficiency of server computations without impacting any other aspects of the algorithm.
B.5.2 A NOTE ON COMMUNICATION COSTS
During each communication round of our algorithm, each client 9 sends K2 arryys of size d and Kg scalars the central server, and receives Kg arrays of size This results per-round total communication cost of approximately 2dGK, , + GK, & 3dGK, g, where K, is the mean number of clusters per client. We compare this to the communication cost of AFCL (Zhang al. 2025). Due to its asynchronous nature, we assume that only 10% of the clients participate in each communication round (a favorable condition for AFCL). In AFCL, each active client sends N,, arrays of size d to the central server, and receives K arrays size d, where K the estimated number clusters. Under the assumption of roughly balanced client sample sizes, it is clear that the total per-round communication cost is approximately 0.IdGN, 0.1dGR 0. 0.1dGN,, where Ng is the mean number of samples per client. Thus, our algorithm enjoys a lower per-round communication cost, since Ng >> 30K, y in most practical applications.
Furthermore, we theoretically prove in Theorems 2 and 5 that our algorithm achieves linear con- vergence rate for all clusters at all clients. In contrast, there is no theoretical convergence rate for AFCL. However, empirical findings (Zhang et al., 2025) suggest a near-linear convergence rate at best. This suggests that our algorithm enjoys lower total communication cost under the setting studied.
C PROOFS
C. PROOF OF PROPOSITION
Proof. Recall that Assumption requires each term in the finite sample (0,00) at client g to be strongly concave, Now, let us define Qny (Ok, 10,)) as follows:
N 2n, (0x, 1))
Σ
Tkg (in, 0 log( Lkg(Tk,Pu Pkg (mn, 0k₂))
Since Mr, M0,(0 (1) is a maximizer of Q2, 10,,10( ((())))), then by strong concavity it must unique.
Therefore, have that Vo., Qr, (0,, (1)-)1) if and only if 0k, Mr, (0(.-1)). As result,
35

---

Published as a conference paper at ICLR 2026
(0) must always be able to obtain unique kg
such that
2h ( !!! (m(00--)))0 & Qu, (M00
vm g )) E e (M >,,.e VE (.).
C.2 PROOF OF THEOREM 1
Proof. Note that Theorem 2 in (Balakrishnan et al.) 2014) only guarantees convergence of the fi- nite sample M-step Mr, (0g) to the neighborhood of the true cluster parameters o but does not examine the behavior within the neighborhood. However, we note that the finite sample EM is still GEM algorithm, albeit characterized via the finite sample expected complete-data log-likelihood function (0,1 Recall that we assume that the function (0,00) is both strongly concave and continuous in both its conditioning and input arguments. Moreover, we assume that the finite sample true oigliikelhoodd is bounded from above. Therefore, by Theorem I in (Wul 1983), the finite sample EM iterates must converge to stationary point of the finite sample true log-likelihood (0)--1)) - 10 ()(-)))
This suggests that (M, 10 9
as t cc. Now, by the
strong concavity of (,,00,) we have that
(0( 1 20000-1000 > M 1) 0 11.
9
where Fg is the strong concavity parameter. This implies that the algorithm must converge single point.
C.3 PROOF OF THEOREM 2
Proof. Observe that we can write the following with probability (1 5g), where 0 & 6g &
|| (0(--1))
0% 112
mk, (0t--1)) MM,, 10) + MM,, (0(t-1)) 0%, 2
(48a)
mk (0(--1)) 1)) Mr, (0 (2- (/)
+ MT,, (0ih-1); 1)
0% 12
(48b)
2
0 Kg
11-1) Mr, (0 (t- (
+
Me,, (0(--1)) 0%
(48c)
Bg (t-1) 1) 0%, 1le
1
unif (N 0g) + | My, (0((
Ag k.
+
Eg Bg >g
(48d)
where (48c) follows from the convergence of the finite-sample EM algorithm. Now, note that by the same argument used in the proof of Theorem 5 we can argue that the term goes to 0 as t - co. This concludes the proof.
MR
C.4 PROOF OF THEOREM 3
Proof. Firstly, note that if ||,,, (0((-1)) 0(221)12 for component kig [Kg] at client [G] diminishes to O at sufficiently fast rate (such a geometric rate for example), then the local iterates m., (0( of our proposed algorithm for the component converge sphere of radius uniff (N,, 6,) centered at the true centroid 0%, with probability of at least (1 (1=5,)- Now,
(1- Xg
in the worst case, the iterates for a specific component & E [K] from all the clients containing this component will converge to some point the surface of the local sphere for each client g. final for all such clients set according to match the radius
Therefore, if the final aggregation radius Ek, of the local neighborhood of the true parameters, then all the aggregation uncertainty sets will also contain gost Therefore, our algorithm recognizes that all these estimates belong one component
36

---

Published as a conference paper at ICLR 2026
and aggregates them. Moreover, the assumption that ER < Bui at all components kg [Kg]] clients g E [G] guarantees that upon convergence, the parameter estimates for different global components k, K E [K] from all clients remain distant enough such that they are not aggregated together. This, however, relies on the iterates for all components at all clients converging to the neighborhood of their true parameters. This is why our proposed algorithm infers the correct number of global clusters with the probability provided in the Theorem statement.
C.5
PROOF THEOREM 4
Proof. We analyze the local client y problem for each component kg as follows.
Jk, (0')
max Ekg
Ekg
N
8. b.
The (iyy : 0²)) log( (os(TT,PH, (In, |mx, (0g)))
=1
N
7kg (Zmr. 0²)) log Pkg (Zng 0k₂,) Vmxx000) B2 (0)); VEKg)
(49)
ERp
Ekg
N₂
S. t.
min
> ng
74g (2ng - 0x₂) 3
mkg
Dng 0%
(50)
Oug (N5 a 10, Me..eee
12u
Me, (0, =+.,
constant
that do not depend on OR.-- and from the fact that Cky O Vkg E [Kg]- Now, let us consider the optimization problem Ma, (0 Ex,) in more
where (50)
follows by ignoring terms in
detail as follows.
min Okg
Σ
7kp (2n, 0')l[ In, 0x 112
Mr, (0' Ekn)
(51)
n2
S.
0kp
Byg (MK, (6')) Ekka
N,
min Okg
Fkg (mng O', (2 Ing 2x Oi, + 0 0rg)
(52)
Reg
8. t.
MK, (00)) - 01, $ cku
min Okg
0kg
Σ
Thi (Zn,, 00)1 0T, +2
Σ 11
ma (Z,, 66, ()))
12
bo
3 ng
The (ing. 00y)1 2i
ng
0k I 0k S.
+
MR, (0,) (MMR,
Mr, (0))
(
MM,(0') Ekg
0,
4
b.
(53)
where A, Rdxd by. b, E Rd, and co. Cj R. Now, note that the above problem is nonconvex, since Ao is not PSD. However, note that for all ek, O, the above problem is strictly feasible.
37

---

Published as a conference paper at ICLR 2026
Therefore, the problem obeys Slater's condition and admits strong Lagrange dual. As shown by Boyd & Vandenberghe (2004), this Lagrange dual can be formulated as the SDP shown next.
max Mky ,kg
Vkg
My, (0 Cky))
== 8.
Okky 3
(54)
An + Ok A1 (bo + an bi)
bo + Okk bj co + Okk C1 - Vka
2 0,
where akgy Mkg E R are dual variables. We can reformulate this problem as follows.
M₁ (0 (g))
max VRg aap
Ukg
8. L.
Ohky 3
(55)
Ao + Oa. A [
bo + Oik b
(bo an. b1) T
co + an c1 1kk
2 0,
max
Ukg
U2g .kkg
S. t.
co + Ok3 C1 - Ukg
(bo + on. Di)T ((ER))
N Σ
(m 0%) 00)))
na
(bo + are b1) (ix,b))
3 0
ng =1
N,
Ohk
Yk, (an, 0²)) 2 0
ng
(56)
max
Vks
8. t.
co + a.2 - Ukg
bo + an, b1113 0
akg
Vkp (Eng 0g)
(57)
N₂
0lk
Tk, (an,, 0¹,) > 0
ng
Vhg
Uky .agg
N₂
s.
(co + an, C1 1kg (ack,
Σ
nh, (Zn, 02) > bb + an, b1113
(58)
-1
N₂ Thy (Ing 0)),
0ikg
> 10
where (56) obtained via the Schur complement. Now, observe that the first constraint in (58) is monotonic in Vk,- Moreover, note that plugging the problem in (58) into the constraint in problem (50) can be interpreted as requiring that the maximum value of Pk, satisfying the constraint must
be greater than equal to - - 7kg (Zng 0z)]lZn, 0¹,,13 113. Thus, it suffices to require that
Uk, EN 7h, (Bn, 0,,)0,0,),,. xr,, 02 11 satisfies the constraint in problem (88 Therefore,
38

---

Published as a conference paper at ICLR 2026
use this result rewrite the problem in 50 follows.
JRa (0')
max
ERg
Ekg 10kg
N2 Σ
Nu
S. t.
Co + c1 +
nh (Zng 12]1 - 8 11
akk
Σ
mkg ( (nng 8/g)
P1 -1
|b0 + an bill3
N₂
3
Σ
Thg (iny: 0¹)
19g
(59)
max Cky
Ekg
N₂
8. t.
Eke kg +
Σ 1ig
The (2n, 0')
-Miy (0') In, In on, + MR, (0))) "Max "M,,('')
-kk, - 1| 02r112 (12) ak,
Ng
Nho (Z 0,))
Σ
Tko (Fmng 03)llarn,
0%,,
My
=y=
Ng
The (zg 0g)
11g
(60)
max
Eku
cng Ss
N₂
s.t.
Ekg akk
Σ
mk (3ng 0¹)) (Ilznn
MR, (0
||x,n
10,112 -Exp)
axk
N
N
Σ
The
(Tn 0,))
Σ
Th (
0 11,
ng 11
11=1
N
0lk Σ
The (Z:n: 0²r)
10g 11
(61)
D SUPPLEMENTARY EXPERIMENTAL DETAILS AND RESULTS
In this section we provide all the details of the experiments presented this paper, as well as supplementary results for both the Benchmarking and Sensitivity Studies. Please note that the all the code and instructions associated with all the experiments is provided separately in the supplementary materials.
D.1 SOFTWARE AND HARDWARE DETAILS
All the experiments presented in this work were executed on Intel Xeon Gold 6226 CPUs @ 2.7 GHz (using 10 cores) with 120 Gb of DDR4-2993 MHz DRAM. provides more detail on all the software used in the paper.
39

---

Published as a conference paper at ICLR 2026
Table Details on All the Software Used in the Numerical Experiments.
Software
Version
License
Gurobi
10.0.1
Academic license
MATLAB
R2021B
Academic license
Python
3.10.9
Open source license
Scikit-Learn
1.2.1
Open source license
Numpy
1.23.5
Open source license
Scipy
1.10.0
Open source license
UCIMLRepo
0.0.3
Open source license
TensorFlow
2.12.0
Open source license
D.2 DATASETS UTILIZED
D.2.1 BENCHMARKING STUDY
In the benchmarking study we utilize various popular real-world datasets for evaluation. We provide more detail on the datasets in Table
Table 4: Details on Datasets Utilized for UCI Experiments.
Datsset
Abbreviation
Lizenee
N
K
MNIST LeCun
2010
CC CCBY4D
Brekmi
(embeddings: MIT License)
Fishion MNIST Auar ai 2017
FMNIST
CC BC 4.0
70,000
10
64
Extended MNIS TRaine (Coten
2017
EMNIST
CC BY 4.0
131,000
47
16
CIAAR-10 (Kuzbevsky
(ombeddinn:: Brckfond Namin
CIFAR-10
(onttedings:: MIT License)
CC BY 4.0
Abalone Noal or 199
Abolose
CC BY
4177
7
Amuran Calls [MFCC Lelonna al
2015
FrogA
CC BY 4.0
7195
10
21
(MPCC))
Frog
CC CC
7195
Wavefomm Datahase Geutstte (Yessol Breieintann
Waveform
Preprocessing Image Datasets. Rather than directly clustering the images in the MNIST, FM- NIST, EMNIST, and CIFAR- 10 datasets, we utilize embeddings extracted from them to reduce the computational expense of the experiments. These embeddings are extracted via variational autoen- coders (VAEs). More specifically, for the MNIST dataset we utilize the vanilla VAE embeddings available at (Bickford Smith et al. (024b), which have dimension 10. For the FMNIST and EM- NIST datasets, we implement VAES with latent dimensions 64 and 16, respectively. Subsequently, utilize the encoded mean vectors of the samples the data utilized for clustering. Finally, for the CIFAR-10 dataset we utilize the "Barlow" embeddings available at (Bickford Smith et al., 2024a). However, further encode these embeddings via VAE with latent dimension 64 as do for FMNIST and EMNIST. All code utilized for feature extraction is provided in the supplementary materials available with the submission.
Abalone. Since the Abalone dataset has very small clusters (some of which contain only 1 sample), combined various clusters together. This makes sense physically, as the target label in the dataset is an integer the age of the abalone. Therefore, combining various labels into bins enforces a more categorical structure on the age. More specifically, we combined labels through 5 into one cluster, kept labels 6 through 10 as separate clusters, combined labels 11 and 12 into one cluster, and combined labels through 28 into one cluster.
Preprocessing
D.2.2 SENSITIVITY STUDY
The data utilized this experiment is generated using the make_blobs module of the scikit-learn Python package. This module generates isotropic Gaussian clusters, making it ideal for our problem setting. The data is generated so that the centroids the clusters have a preset minimum distance of Rmm between them. Moreover, data generation is designed two of the generated clusters have centroids that are exactly Rmin apart. It is worth noting that for the sensitivity study, the dataset generated during each repetition is tested 3 times for each model.
that at least
40

---

Published as a conference paper at ICLR 2026
During each of those times, the model starts with random initialization via x-means++. The results we report are the maximum performance obtained over the 3 initializations.
D.3 HYPERPARAMETER DETAILS AND PERFORMANCE EVALUATION
In this section, we provide a detailed practical discussion on the tuning our algorithm's 's final aggre- as a practical heuristic, where Vy gation radii. As mentioned Section 6, we use Ek, The VrvV
Vg Raing
giinall
is the hyperparameter we directly tune. For simplicity, we set Un equivalently for all clients E [G]. This heuristic allows the aggregation radii to scale appropriately with the scale the feature space and the number of samples available at each client. Additionally, it allows the final aggregation radii to adapt to each cluster at each client while requiring the tuning of only one hyperparameter.
We utilize cross- coss-validation to tune Vg.. using SS Rousseeuw (1987) as performance metric. We also provide practical guide to evaluate the estimated K' without knowledge of the true K. To that end, we ensure that K does not significantly exceed v VEg- Kg. This guide inspired by rough estimate that for any client E [G]. Kg ~ O(K), which suggests that Σ Kg (OKG)). Since in most practical cases we have G > K, then we can see that v - K, O(VKG) > O(K).
All hyperparameters for all benchmark models are set as prescribed in their respective works. Ad- ditionally, note that the estimated number of clusters provided to the DP-GMM and AFCL models is Σg Kg, as this constitutes an upper bound for K. Note that this initial estimate significantly
NG N₂ suggested for 32
closer to the true number of clusters for all datasets than the initial value of AFCL in Zhang et al., 2025). The reported estimated number of clusters for both algorithm is the total number of clusters to which test samples were assigned. Moreover, we run all iterative algo- rithms for T 20 iterations, and we run algorithm for T 10 iterations. Furthermore, utilize local steps for our model in all setting, as well as 10 iterations for Algorithm [
It should be noted that since model is personalized, the reported performance for our model is weighted average of the clients' individual performance metrics.
Table 5: Hyperparameter 0g tuning values for all datasets used in the Benchmarking Study.
Dataset
Hyperparameter Value(s)
MNIST
2e0
FMNIST
3c2
EMNIST
2e0
CIFAR-10
2e1
Abalone
(5e3, 7c3, 9e3)
Frog A
(1e4, 1e5, 1e6)
Frog B
[le4, le5, le6)
Waveform
(le0, 5e0, lel]
Synthetic
{5e-
1, 1e0, 5c0, le1, 5c1]
D.4 SUPPLEMENTARY BENCHMARKING STUDY RESULTS
D.4.1 SILHOUETTE SCORE
We provide additional results for our Benchmarking Study using the SS evaluation metric in Table 6 We immediately observe that our proposed method continues to attain the highest performance out of the federated methods with unknown K for most datasets. Furthermore, we observe that on datasets such as Abalone and Waveform, our method outperforms even the top performing method with known K. Since the results using these evaluation metrics similar those using the ARI, we reach strong conclusion that our proposed model has a significant practical impact. Namely, can achieve similar performance to, even outperform some clustering methods that assume prior
41

---

Published as a conference paper at ICLR 2026
knowledge of K, and it often outperforms method without prior knowledge of K. It achieves this while being federated (i.e. not requiring any data movement), and without prior knowledge of K.
Table 6: SS attained by all methods on tested datasets.
Model
Known
MNIST
FMNIST
EMNIST
CIFAR-10
Abalone
Frog
Frog
Waveform
GMM (central)
Yes
L.020
.029
1.008
.093
1.006
.046
1.013
.191
1.014
.397
1.042
.232
1.089
.198
1.026
.55
K-FED
Yes
.076
.106
.075
.177
408
.285
.278
.245
= =.010
f.022
+.010
f.012
+. 4.0
+. 0.0
076
028
-045
.009
-1133
-.014
400
.29
.259
.217
FFCM-avg!
Yes
i. 2.030
i.
1.045
1.051
FFCM-avg2
Yes
.036
.052
-.077
.087
.04
.272
.324
.231
=.011
2.018
+2023
+.021
0.016
0.071
.50
042
.105
.127
.991
203
404
302
280
.222
Yes
= 5.005
+ 2006
+ 003
t 006
+ 4.010
0.054
2.074
003
DP-GMM (central)
No
1.014
.082
1.014
- .058
1.007
-,063
1.005
.177
d.023
.24
1.040
.71
1.025
.144
115
.015
.017
.288
.066
.192
.144
.156
.018
AFCL
No
f .002
2.003
0.002
1.005
1.028
1.048
2.045
0.0
FedGEM (ours)
.095
.069
.063
.091
307
.344
.844
.211
1.012
1.018
1.009
1.015
1.086
1.082
1.092
1.011
D.4.2 BENCHMARKING AGAINST MODELS WITH KNOWN K VIA ESTIMATION HEURISTIC
We perform an additional study where we compare the performance of our FedGEM algorithm to that of one with known K, where K is treated as hyperparameter. We note that all results presented this study are averages over 20 repetitions.
Our Method. As with all the previous experiments, our method is the isotropic GMM from Section 5 trained via our FedGEM algorithm.
Evaluation Metrics. In this experiment, we evaluate model performance based on ARI and runtime. This allows us to examine potential improvements offered by our approach terms of clustering performance as well as the potential sacrifice in computation needed to achieve this improvement.
Baselines. We compare our algorithm to FedKmeans Garst Reinders (2024) due to its high performance in the Benchmarking Study in Section 6 which makes it strong baseline.
Dataset. We leverage dataset generated via the make_ baoke_boob module in Python with K 20 and Ryin 2 across 4 distinct settings. Namely,
vary the maximum amount of clusters K g.m available at any one client such that Kgm. E (5, 10, 15, 203. Note that during each repetition, there need not be client g whose Kg K gom Rather, gem is upper bound on K, for all g [G], which may or may not be attained based on the randomly assigned clusters. all settings, we use G 15 clients, = 4 features, N, 400 training samples per client, and N 2000 test samples.
Hyperparameters. In this experiment, tune the hyperparameter vn associated with our model within the values Vi E (1e0, 5e0, 9e0, lel, 5e]]. Similarly, we tune K for FedKmeans. To that end, we search over 5 equally spaced values of K, the smallest of which is the maximum amount of clusters K available at any one client, and the largest which the sum N.G Kg. These values are chosen as they form lower and upper bounds through 3-fold cross-validation using the SS as metric due to lack of labels during training time. Additionally, we note that we ran FecKmeans for 10 iterations, while we used S iterations and T = global round for algorithm.
respectively. All tuning is performed
local
Results. We present the results of this study Figure 2 Firstly, we observe that our approach offers much stronger and more stable clustering performance across most settings, with much smaller standard deviation. This is because FedKmeans's performance is very sensitive to the tuning values of K, whereas our algorithm's methodical inference of the cluster number allows to enjoy improved stability across hyperparameter values and problem settings. Secondly, we observe that the performance of approach slightly diminishes K gas increases. This could be
42

---

Published as a conference paper at ICLR 2026
Clustering Performance
Runlime
10'
0.6
Ramtime
Ant
10a
0.4
0.2
10a
0.25
0.5
0.75
0.25
0.5
0.75
K
RelKmoons
ReaGEM (Uuas)
K
Figure 2: Results of the study comparing our proposed algorithm to one with prior knowledge of K where K is treated as model hyperparameter.
attributed to two key reasons. First, we use fixed hyperparameter tuning values across all problem settings, whereas different values may lead to improved performance for higher K gess Secondly, the personalized nature of performance evaluation in our algorithm may result in slight performance reductions as the number of locally available clusters each client increases. On the contrary, we observe a general increasing trend in FedKmeans 's performance as K gen increases. This could be attributed the smallest tuning value of K approaching the true value of K 20 as K 9ma increases. Indeed, we observe very strong performance at K yes = K = 20, as the true number of clusters is included in the tuning values. However, FedKmeans's 's performance with K 9i.. 10 highlights the potential problem with the use of such tuning heuristic for methods that assume prior knowledge of K. Namely, the selected tuning values may be highly incorrect, which can result in very poor performance even with the use of cross-validation. Similarly, we observe that the standard deviation of F FedKmeans' 's performance at Kg... = is very large. This is likely due to very different values of the hyperparameter K being selected by the vrossvvaliaation heuristic across different repetitions, resulting in very large performance variations and instability. Finally, we observe that the improved performance and stability offered by our approach comes very minor additional computational expense while maintaining the same scalability as FedKmeans. We note that while these results are promising, they may differ with different datasets. This motivates future work carefully study the conditions under which our proposed algorithm is advantageous over algorithms with known K, where K' is tuned as a hyperparameter.
D.5 SUPPLEMENTARY SENSITIVITY STUDY RESULTT
We present the results of the Sensitivity Study utilizing the SS to compare model performance in Figure 3 Firstly, we again observe that performance both models improves as Rmin increases. Surprisingly, however, we see that our proposed model outperforms GMM in all setting, which does not match the ARI result. This could be explained by SS's sensitivity to the number Indeed, problem with a smaller number of clusters is likely to exhibit higher SS than an identical one with a larger number of clusters. Since each client only has a subset of the clusters locally, this can cause the local SS to be over-inflated. However, our proposed model offers very close performance to that of a centralized one with known K, which is powerful result.
clusters.
in the ARI result, we conclude that
D.6 SENSITIVITY TO HYPERPARAMETER
This study evaluates the sensitivity of our proposed algorithm to its final aggregation radius hyper- parameter.
Our Method. As with the other numerical experiments, our method is the isotropic GMM model trained via our proposed FedGEM algorithm.
Evaluation Metric. We examine the sensitivity of both the ARI and the estimated number of clusters to the hyperparameter.
43

---

Published as a conference paper at ICLR 2026
6.5
Nociel
2.5
Clder Inbosaree
0.5
Chette Inbaksan
S0.ucle Sout
500 Dil-oattt
03
(a) Silhouette Score for our proposed FedGEM centralized GMM trained via EM.
Nemaan
Clet
(b) Number of clusters estimated by our proposed FedGEM vs. the true number of clusters.
Figure 3: Supplementary results of the sensitivity study.
Hyperparameters. Recall our final aggregation radius heuristic kk Vig Ruii We evaluate our model's performance for Un E (le- 1, 1e0, le2).
Dataset. The data used for this experiment is isotropic Gaussian clusters generated via the make_blobs module in Python. We set Rmin 4, and we study three key settings: nomi- nal: data is balanced across clients and clusters, ii) client imbalance: the data is imbalanced across clients, and iii cluster imbalance: the portion of each cluster the local data at each client is ran- domly samples followed by normalization. For all settings use G 15, K 10, N Nuain 7500, and Nis = 2000.
The results of this study displayed in Figure 4 We observe that the estimated number of clusters can be more sensitive to the choice of Vg than ARI. This intuitive, as a value of Ug that is too small will result in insufficient cluster aggregation, which causes the estimated number of clusters to be overinflated. However, since clustering performance evaluation is performed locally each client, ARI can still be somewhat stable this setting. On the other hand, v., is too large, this will cause estimates associated with different clusters to be aggregated together. This leads to an underestimation of the number of clusters and also significantly affects clustering performance. A key observation we make is that for an appropriately adjusted Ug, ARI seems to reach peak value in the nominal case while the estimated number of clusters almost coincides with the true value. This highlights the importance of hyperparameter tuning via the protocol we present in Appendix [D.3 Finally, we note that the cluster and client imbalance settings do not significantly affect our model's performance, suggesting robustness such issues.
Results.
E SCALABILITY STUDY
In this section, we present two distinct studies we performed to evaluate the scalability of our pro- posed algorithm.
Our Method. As stated in the full paper, our method is the isotropic GMM model trained via our proposed FedGEM algorithm.
Evaluation Metric. We utilize runtime in seconds to evaluate the scalability of all methods.
Baselines. We compare our algorithm to AFCL, which is the only other federated clustering method that does not require knowledge of K. Additionally, we also use FFCM-avg2 and F aedkmeans as benchmarks as they achieved strong performance in the Benchmarking Study.
44

---

Published as a conference paper at ICLR 2026
Nounizal
Cleat Inbalance
Cluster Inbolance
0.8
0.8
0.5
A00
0.6
0.6
0.5
10
120
100
10
105
(a) Sensitivity of our algorithm's clustering performance measured via ARI to the hyperparameter value.
Nocinal
Inbolanee
15
Kunber
10a
107
20a
10
227
10"
107
(b) Sensitivity of our algorithm's number of clusters estimation to the hyperparameter value.
Figure 4: Results on the sensitivity of our algorithm to its hyperparameter.
E. SCALABILITY ON IMAGE DATASETS
We examine the runtime of some our algorithm as well as some of the federated benchmarks on the larger-scale image datasets. This allows evaluate the scalability of proposed algorithm in realistic settings.
Hlpeepaaaaeeeer. All hyperparameters and experimental settings (e.g. number of clients G. hy- perparameter settings, etc...) are exactly the same as described in detail in Appendix [.3] However, in the interest of fairness, their performance after training is on par with the values reported previously.
all federated algorithms for T 10 iterations, and confirm that
Datasets. In this experiment focus solely on the MNIST, FMNIST, EMNIST, and CIFAR-10 datasets. This is because they are on a much larger scale than the other datasets tested, therefore they provide meaningful insights into algorithm scalability.
Results. The results of this experiment are reported in Table 7 We observe that our algorithm achieves a much shorter runtime than AFCL (the only other federated clustering approach without prior knowledge of K). This emphasizes the significant practical impact of our algorithm, as it also achieved superior clustering performance and total number of cluster estimation as discussed in Section 6 and Appendix D As discuss in the Scalability Study Synthetic Data in Appendix E.2. this advantage over AFCL is most likely due to improved scalability with respect to the number of clients. This suggests that our algorithm is better suited for distributed clustering problems over large networks involving large volumes data.
Table 7: Runtime seconds of selected federated algorithms on the image datasets evaluated.
Model
Known K?
MNIST
FMNIST
EMNIST
CIFAR- 10
FFCM-avg2
Yes
220 + 18
140 + 25
6075 = 842
188 8 +
FedKmeans
Yes
30 +
52 3
314 =
AFCL
No
2047 + 246
2013 + 204
3176 = 722
1798 + 165
FedGEM (ours)
No
552 +
645 + 63
1628 335
345 + 35
E.2 SCALABILITY ON SYNTHETIC DATA
This study aims to evaluate the scalability of our proposed algorithm the size of the training dataset and the federated network grow. It also compares the scalability our algorithm to that of multiple federated benchmarks. We note that the implementation of algorithm used in this study relies on pairwise server computations. Therefore, scalability can likely be further improved by leveraged KD tree explained in Appendix B.5
45

---

Published as a conference paper at ICLR 2026
Hyperparameters. Since the focus of this study is more execution time than model perfor- mance, we did not perform hyperparameter tuning for this experiment. We fix our final aggregation radius hyperparameter U/ 1e0 for all g G. We set the hyperparameters of benchmark models as preseribed in their corresponding papers. Additionally, we run all algorithms with 7 10 iterations. Finally, this experiment was repeated for 10 repetitions.
Dataset. In this experiment we utilize data generated via the make_bl blobs module in Python, which generates isotropic Gaussian clusters. We utilize Rmin 2 across all experiments. Addition- ally, we study 4 distinct experimental settings, listed next.
1. Increasing Features: G = 5, N, 500, K = 10, de (5, 25, 45,65, 65).
2. Increasing Training Samples per Client: G = 5, K = 10. d = 15, No (500, 2500, 4500, 65003.
3. Increasing Clusters: G == 5, N₂ = 500, 15, K E (5, 25, 45, 65).
4. Increasing Clients: = 1000, K 10, d 15, G E (5, (5, 45, 65).
Across all experiments, we uniformly sample K2 for all g E [G] such that 2 E Kg < K.
Results. The results of this experiment are shown in Figure Firstly, we observe that the runtime of all algorithms remains constant as the number of features increases. This suggests that all com- pared algorithms, including ours, scale well with the number of features. Secondly, while our algorithm exhibits a greater runtime than benchmark methods in all experimental settings, it scales similar rate them the number of samples, clusters, and clients increase. This sug- gests strong scalability across all settings. Moreover, we observe in the increasing number of clients setting that AFCL's runtime increases at a faster rate than our proposed algorithm. This suggests that our algorithm scales better in this setting, and can therefore be more suitable for settings with a large number of clients. This observation aligns with our results presented in the runtime analysis in Appendix[E.. where we observe that our algorithm achieves a shorter runtime than AFCL in exper- iments involving a high and large datasets. Combined with the fact that our algorithm exhibited better clustering performance and true number of clusters estimation across all our experiments, this highlights the significant practical impact of our proposed FedGEM algorithm.
observe that
Festures
Samples per Client
10
10
102
1
Runtins
Runtime
100
20
10
500
2500
4500
Festures d'
# SSnaples per Cleut N,
Clastees
Clienta
10
2
Rostinu
10a
Ruztine
10a
2
10a
25
45
65
25
45
65
Clusters K
# Clliects
aog)
Figure 5: Results of the scalability experiment for all experimental settings and benchmark models.
46

---

Published as a conference paper at ICLR 2026
F FURTHER DISCUSSION
F.1 JUSTIFICATION AND INTERPRETATION OF MODELING ASSUMPTIONS
Assumption I Ground Truth Parameters. In this assumption, we enforce a modeling structure that necessary for the convergence analysis of our algorithm. Namely, that any clusters that are shared by multiple clients, have the exact same ground truth parameters at all clients. Note that this assumption does not violate the non-IID nature of the data in FL problems. This is because cluster weights can be different across clients, and clients may have different clusters. Therefore, the data across clients still non-IID. This assumption is common in works studying federated EM algorithms, such as (Marfoq et al. (021).
Assumption 2 Strong Concavity. This assumption requires each of the terms in the expected complete-data log-likelihood functions to be strongly concave, thereby allowing for the function to have a unique maximizer. Such assumption is very common (at least locally near the optimum) in works examining the convergence of EM algorithms such as (Balakrishnan et al. 2014). This assumption is also readily verifiable for models such as GMMs.
Assumption 3 First-Order Stability. This assumption requires the expected complete- data log- log-likelihood to obey Lipschitz-like smoothness constraint, introduced by |Balakr ishhan et al. (2014) and defined in [ Such a technical assumption is vital for the theoretical analysis, and the derivation of convergence guarantees, but is not required for the algorithm to be used in practice.
Assumption 4 Continuity. This another technical assumption, which requires the complete- data log- liglikeli function of the model used to be smooth in both its input and conditioning arguments. This is standard in many EM-related efforts, and is only required for the theoretical convergence analysis but not for algorithm use in practice.
Assumption 5 Likelihood Boundedness. This assumption requires the log- likelihood of the model used to be bounded from above, although that bound need not be known. Similar to previous assumptions, this one is also purely required for the theoretical convergence analysis, but not for the use of the algorithm in practice. Note that this assumption is common in works investigating Federated EM algorithms such as (Marfog et al. 2021), and is easily verifiable for models such as GMM under mild conditions the covariance matrix.
Assumption 6 Finite-Sample and Population M-Step Proximity. This assumption quires there to be an upper bound on the maximum difference between the population M- step and the finite-sample M-step for each cluster with certain probability. Whereas all the previous assumptions allow us to theoretically study the convergence of our algorithm on the population level (i.e. with infinite data), this one necessary for the sample convergence analysis. Specifically, it allows us to prove that the algorithm updates made via a finite data sample indeed converge to a neighborhood of the converged population- based iterates. This assumption was utilized in works exploring the convergence of EM algorithms such as (Balakrishnan et al,, 2014 Yan al. 2017), and is also purely technical and does not impact algorithm usability in practice.
Assumption 7 Bounded Support. This assumption requires the support of the feature vector be bounded, and is needed only in the setting where DP is used to privatize the cluster maximizers shared by the clients. Such an assumption is not restrictive. This is because data is often collected via acquisition devices with known ranges. Therefore, feature support is either already bounded, or can be via normalization.
F.2 INTERPRETATION OF THEORETICAL RESULTS
Proposition I Local Uncertainty Set Radius Problem. This proposition asserts that the optimization problem solved by each client to obtain the radius of the uncertainty set centered at the maximizer of each local cluster must have unique solution. The unique solution would be 0 at convergence. This holds under the modeling assumption thanks to the strong concavity of the complete-data log-likelihood function.
47

---

Published as a conference paper at ICLR 2026
Theorem I Single-Point EM Convergence. This theorem asserts that the finite sample EM iterates computed by each client for each local cluster must converge to a single point withing certain proximity of the ground truth parameters. This is subtle, but key result, as ensures stability and lack of oscillations upon convergence.
Population GEM. This theorem asserts that, in the population setting (i.e. infinite training samples), iterates that are computed via our pro- posed FedGEM algorithm converge exactly to the ground truth parameters. This is a very strong convergence result, which is used to establish the saite- convergence of the algorithm.
Theorem 5 Local Convergence
Theorem 2 Local Convergence of Finite-Sample GEM. This theorem asserts that, with a certain probability, iterates that are computed via our proposed FedGEM algorithm con- verge within certain radius around the ground truth parameters at any client. This is achieved with only a finite number of training samples. This result forms the basis for our convergence argument. This is because iterates of a shared cluster across multiple clients converge close proximity of each other. Therefore, given final aggregation radius that meets certain conditions, they can be successfully aggregated into a single cluster.
Theorem 3 Number of Clusters Inference: This theorem asserts that with certain probability, our algorithm correctly estimates the total unique number of clusters across clients. This is reliant on the finite-sample convergence established Theorem P
Theorem 4 Radius Problem Reformulation. This theorem provides tractable, bi- convex, 2-dimensional reformulation for the semi-infinite uncertainty set radius problem in the case of isotropic GMMs. This renders our algorithm tractable for this specific model, and allows us to implement it in our numerical experiments.
Proposition 2: Local Radius Algorithm Convergence, This proposition shows that Algo- rithm 4 proposed to solve the uncertainty set radius problem reformulation from Theorem Menjoys a very low time complexity. This allows our FedGEM algorithm to scale well with problem size.
Theorem 6: GMM First-Order Stability. This theorem proves that the multi-component isotropic GMM explored in this work indeed satisfies the FOS condition defined in [ This is very impactful result, as, to the best of our knowledge, this is the first time such result is formally proven for a GMM with more than two components. This condition is necessary for the convergence of our algorithm. Therefore, formally proving it allows to argue that our F FedGEM algorithm is guaranteed to converge for multi collii-ommpoentt, isotropic GMMs.
Theorem 7 GMM M-Step Contraction Region. This theorem examines the necessary problem conditions, including an upper bound on the radius of the contraction region at each client, for the multi-component isotropic GMM to converge. This discussion allows to argue that proposed algorithm converges for the isotropic GMM under considera- tion if the problem conditions are satisfied. However, we note that this is a purely technical result needed only for the thcoretical convergence analysis, but not for practical implemen- tation.
Theorem g GMM Finite-Sample and Population M-Step Distance. This theorem de- rives the upper bound on the distance between the population and final sample M-steps that is required by Assumption (6] The existence of this bound guarantees the convergence of our proposed FedGEM algorithm for the isotropic GMM under study. Note, however, that this also a purely technical result required only for the theoretical convergence analysis. However, it is not needed for use of our algorithm in practice.
Theorem 9 Client-to-Server Communication DP. This theorem is provided as part of a preliminary DP discussion. provides the minimum standard deviation of the Gaussian noise to be applied to the maximizers shared by the clients to guarantee DP.
F.3 LIMITATIONS AND FUTURE WORK
This paper lays the foundation for a wide array of future work that can provide significant con- tributions and advance the fields of clustering, federated learning, and unsupervised representation learning via mixture models. Next, we discuss some of the limitations of our work, which should be addressed in future work.
48

---

Published as a conference paper at ICLR 2026
Fixed Cluster Weights. While our algorithm allows each client to set personalized local weights for their local clusters, these weights are fixed. In order to enhance modeling flexibility and personalization capabilities, future efforts should extend our algorithm to include trainable local cluster weights.
Stylized Clustering Model. While the FedGEM algorithm we propose generic, we mainly focus its with an isotropic GMM in this work. Future work may improve real-world performance by utilizing our algorithm with more complex mixture models, potentially studying anisotropic GMMs with locally learnable cluster weights. This would be theoretically challenging as it would involve verifying the needed assumptions, as well deriving tractable formulation for the local radius problem. Moreover, this may require an alternative convergence analysis approach, such as one that focuses on convergence to stationary points rather than (neighborhoods of) global maximizers. Furthermore, such efforts would need to study how the use of such complex models impacts the aggregation process the central server.
Differential Privacy. While do provide preliminary discussion privatizing the cluster centroids shared by each client via DP in Appendix [3.4] privatizing the uncertainty set radius and studying convergence in more detail remains an open problem. Since the radius is computed via an optimization problem, key theoretical contribution would be analyzing its sensitivity and deriving the appropriate DP budget.
Modeling Assumptions. In order derive theoretical convergence guarantees for F FedGEM algorithm, we make various modeling assumptions. While these assumptions are very commonly made, and do not severely impact performance in practice if they violated as shown in Section 6 a valuable contribution would still be deriving convergence guarantees with relaxed assumptions.
Pairwise Server Computation. In proposed work, the server relies pairwise com- parisons between the clusters at all clients in order to infer overlaps. While we have shown in our Scalability Study in Appendix E that our algorithm scales well with problem size, scalability can be further improved. Future work may develop a more efficient algorithm be used by the central server to infer cluster overlaps.
Full Client Participation. While the proofs presented in this work would still hold under partial client participation, they do not account for the potential drift that can be experienced by stragglers. While convergence a neighborhood of the global maximizer is proven for centroid estimates at all clients, client drift can cause the estimates to end in relatively distant areas of that neighborhood. This can increase the sensitivity of the estimated number of clusters to the final aggregation hyperparameter. Future work may study this setting both from the theoretical and practical perspectives, providing stricter convergence guarantees for stragglers and potential strategies to ensure an accurate estimation of K.
Final Aggregation Radius Tuning. While we present reliable heuristic and a guideline that can be used to set the final aggregation radius in our algorithm, still requires hyper- parameter tuning via cross cross-validation to exploit our algorithm's 's full performance potential. Such tuning can ineur very large computational costs, and can also significantly affect DP guarantees. Future work may seek to explore more robust, data-driven and theoretically verified hcuristics that can achieve near-optimal performance while minimizing the com- putational and privacy costs associated with cross- volsdaaaia based tuning.
49


Reviews:
[{"review_id": "J5t4zWwnc0", "reviewer": "Reviewer_aUfD", "summary": "This paper proposes FedGEM: a federated generalized expectation-maximization algorithm for the training of mixture models with an unknown number of components. The algorithm relies on each of the clients performing EM steps locally, and constructing an uncertainty set around the maximizer associated with each local component. The central server utilizes the uncertainty sets to learn potential cluster overlaps between clients, and infer the global number of clusters via closed-form computations. \n\nThis paper performs a thorough theoretical study of our algorithm, presenting probabilistic convergence guarantees under common assumptions. Subsequently, this paper studys the specific setting of isotropic GMMs, providing tractable, low-complexity computations to be performed by each client during each iteration of the algorithm, as well as rigorously verifying assumptions required for algorithm convergence. This paper also performs various numerical experiments.", "questions": "As above.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The structure of the paper is relatively clear. \n2. The theoretical derivation and proof in the paper are quite thorough.", "weaknesses": "There are only two compared algorithms that do not assume prior knowledge of K. Among them, one is the algorithm proposed in 1974, and the other is AFCL, which uses entirely different datasets from those in this paper. These two aspects result in flaws in the experimental design, making it difficult to truly reflect the algorithm's performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "54B9KgW7vX", "reviewer": "Reviewer_iREx", "summary": "The paper introduces FedGEM, a federated generalized EM algorithm for mixture models with an unknown global number of components (\\\\(K^\\*\\\\)). Each client runs a local GEM on an isotropic GMM with fixed mixture weights and covariances, updating only component means and computing uncertainty radii. The server merges overlapping components based on these uncertainty sets and performs constrained parameter updates. Theoretical results establish finite-sample and population convergence, as well as a sufficient condition for correct (\\\\(K^\\*\\\\)) recovery. Experiments on synthetic and benchmark datasets show competitive clustering accuracy and moderate scalability.", "questions": "See weaknesses.\n\nOther questions:\n\nAnisotropic models: Do you foresee obstacles to extending the convergence and finite-sample analysis to full-covariance Gaussians (or even tied/diagonal covariances)? What breaks in the proofs?\n\nIs fixing pi essential for your bounds, or could pi be updated (with constraints) without derailing the strong concavity/FOS arguments?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "# originality\nFederated EM with unknown global K via uncertainty-set–based merging at the server is an interesting angle relative to standard federated clustering/EM, and distinct from k-means-style approaches\n\n# quality\n\nThe paper presents local convergence results for the GEM variant, as well as finite-sample versus population map deviation bounds under the stated assumptions.\n\nthe radius subproblem has a stated unique solution and a low-complexity solver with a convergence and complexity discussion\n\n# clarity\n\nProblem setup and the server pseudo-code for super-clustering/aggregation are explicit\n\nAssumptions are listed and some are justified in an appendix\n\n# significance\n\nUnknown K across many clients arises in FL. The approach could be a useful building block if the server-side merging is robust and the hyperparameters are chosen well. Empirical results indicate competitive performance and a reasonable scaling trend", "weaknesses": "# originality\n\nThe method is positioned generally (\"mixture models with unknown K\"), but the analysis and implementation hinge on isotropic Gaussians with fixed weights; this narrows the contribution relative to the stated ambition. Extending to anisotropic covariances or learning pi would be more compelling.\n\n# quality\n\nweights fixed, covariance fixed to identity, Kg known locally. the paper also does not study the effects of mispecified pi.\n\nThe overlap-detection step entails pairwise checks over all client components, i.e., quadratic in the total number of components. the paper relies on empirical timing rather than giving a tight complexity analysis for this stage. This should also be discussed as a limitation.\n\nit is not analyzed whether a client (or all clients) stuck at a sharp local maximizer will \"snap back\" after the server’s within-set update. \n\n\n\n# clarity\n\nAssumption 6 (supremum variable vs. the conditioning argument of M / $\\hat{M}$\nThe role and definition of the strong concavity parameter ​$\\lambda_g$ are not introduced where first used (Assumption 3)\n\n# significance\n\nFinal aggregation radius $\\epsilon^{final}$ is a user hyperparameter. While a sufficient condition wrt Rmin is given, Rmin is not known. The paper should indicate a protocol for choosing this hyperparameter\n\nNo discussion of communication/computation trade-offs vs. alternative K estimation strategies (e.g., centralized model selection surrogates, Bayesian nonparametrics) in FL.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pSOSS3jJGO", "reviewer": "Reviewer_BFt3", "summary": "This submission presents a federated generalized expectation-maximization (FedGEM) algorithm for clustering in a setting where clients have heterogeneous data and the total number of clusters across the clients is unknown. FedGEM involves clients performing local EM steps to identify local cluster centers and then constructing uncertainty sets around these centers. These sets (centers and radiuses) are communicated to a server, which infers overlaps between local clusters to collaboratively train shared cluster parameters and estimate the possible total number of clusters. The authors provide a theoretical analysis of the convergence of the FedGEM and its ability to correctly infer the cluster count. The empirical evaluation demonstrates that FedGEM achieves state-of-the-art performance, outperforming the mentioned baselines.", "questions": "- How to set `final_aggregation_radius`? Could you provide more practical heuristics for setting this value? How sensitive are the ARI and the estimated K to this hyperparameter?\n- Others, please refer to weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Sound Validation: The extensive experimental results demonstrate the effectiveness of the FedGEM. The results in Table 1 show that FedGEM outperforms AFCL, the only other federated baseline that operates without knowing K. Also, FedGEM is competitive with federated methods that are given the true K in advance. The sensitivity study further indicates the robustness of FedGEM, showing strong performance even when theoretical assumptions like well-separated clusters are violated.\n- Theoretical Analysis: This submission is supported by a theoretical analysis. The authors provide probabilistic convergence guarantees for the general FedGEM algorithm under standard assumptions (Theorems 1, 2, and 5). Also, this submission provides a detailed analysis for the isotropic GMM setting, where they prove the First-Order Stability (FOS) condition for multi-component GMMs (Theorem 6).", "weaknesses": "- Generality: The implementation, theoretical verification, and experiments are based on an isotropic GMM. However, its performance on real-world data with arbitrarily shaped clusters might be limited.\n- Computational Complexity: The server is required to identify cluster overlaps involves pairwise comparisons between all local components from all clients. This is a computational complexity that scales quadratically with the total number of local components across the network. Though the scalability study in the appendix shows good performance, this could become a bottleneck in FL settings with thousands of clients.\n- Presentation and Grammar Issues: 1) grammar issue: line 96-97; 2) the names of the mentioned methods should be in the same font if the authors would like to emphasize", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ao0HyuasLo", "reviewer": "Reviewer_vQdW", "summary": "The authors discuss clustering in the federated setting. Data is distributed across clients, who share cluster centroids but follow different cluster weights, specifically, no client has data from all clusters. . The effective novelty of the paper is the ability of the proposed FedGEM to automatically infer the number of clusters across the federation. \nThe authors discuss convergence properties of FedGEM in the general case and for the special case of GMMs. The paper concludes with a variety of empirical studies, specifically showing that FedGEM works even if \"real\" data is not well-separated, thereby invalidating a core prerequisite for the theoretical analyses (concavity).", "questions": "If the authors could discuss the limitations I believe to be missing in the paper and also discuss communication costs, I'll consider raising my score!", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper appears to be thorough and well-executed. The authors spend time analyzing a lot of aspects of their method in an extensive Appendix. The methods limitations are well-covered along with its strengths.\nThe problem of federated clustering is highly relevant, the method is original and automatic cluster count detection unlocks a new capability. \nNote that I did not check derivations.", "weaknesses": "I do not follow the paper's motivation about OEM fault detection. How exactly does this relate to (federated) clustering? The authors claim the dimensionality of the data poses a problem - however all of their experiments are well within the range of internet-transferable sizes. (The largest dataset across clients comes in at ~17mb). \nAlthough the paper describes some limitations of the proposed method, I see a few more that I think need to be addressed:\n- The entire analysis and experimental result assumes full client participation. A realistic federation might only have a subset of clients participate at every round. The server-side aggregation would require rethinking\n- The final aggregation radius needs to be determined through cross-validation. This is highly impractical in the federated setting as it would involve running the entire algorithm end-to-end multiple times. Furthermore, the optimal radius depends on the data geometry, scaling the feature space would require a new radius.\n- It is unclear how anisotropic covariances would change the nature of aggregation radii\n- Cross-validation incurs a huge price for DP guarantees\n\nmissing communication cost quantification. The authors note that AFCL requires clients sharing arrays of the same size as the local data. That price might be worth wile to pay (in terms of communication cost) if the total number of communication rounds is smaller due to a faster convergence rate. Especially as the number of clusters per round is variable, per-iteration costs of transmitting centroids could be high.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)