Title: OmniFC: Rethinking Federated Clustering via Lossless and Secure Distance Reconstruction
Venue: NeurIPS 2025
Accepted: yes
Keywords: Secure Federated Clustering

Non-IID Data

Lagrange Coded Computing

Distance Reconstruction
Abstract: Federated clustering (FC) aims to discover global cluster structures across decentralized clients without sharing raw data, making privacy preservation a fundamental requirement. There are two critical challenges: (1) privacy leakage during collaboration, and (2) robustness degradation due to aggregation of proxy information from non-independent and identically distributed (Non-IID) local data, leading to inaccurate or inconsistent global clustering. Existing solutions typically rely on model-specific local proxies, which are sensitive to data heterogeneity and inherit inductive biases from their centralized counterparts, thus limiting robustness and generality. We propose Omni Federated Clustering (OmniFC), a unified and model-agnostic framework. Leveraging Lagrange coded computing, our method enables clients to share only encoded data, allowing exact reconstruction of the global distance matrix—a fundamental representation of sample relationships—without leaking private information, even under client collusion. This construction is naturally resilient to Non-IID data distributions. This approach decouples FC from model-specific proxies, providing a unified extension mechanism applicable to diverse centralized clustering methods. Theoretical analysis confirms both reconstruction fidelity and privacy guarantees, while comprehensive experiments demonstrate OmniFC's superior robustness, effectiveness, and generality across various benchmarks compared to state-of-the-art methods. Code will be released.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nOmniFC delivers a drop‑in federated clustering method: clients split their quantized feature vectors into segments and compute Lagrange‑coded evaluations over a finite field, which they send to the server. \nUsing only these coded values, the server can reconstruct the global pairwise distance matrix $D_{ij}=\\|x_i-x_j\\|^2$ losslessly and then applies standard k‑means, spectral clustering or DBSCAN without ever seeing raw data. Theorem 1 shows that $m\\ge2l+2t-1$ evaluations suffice for exact recovery, and Theorem 2 establishes information‑theoretic $t$-privacy when $m\\ge2\\ell+2t-1$. Empirical results under extreme non‑IID splits demonstrate that OmniFC matches centralized Kappa and NMI while drastically reducing communication and compute costs.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1.**Theoretical Proofs.** The authors provide comprehensive theoretical proofs for OmniFC, demonstrating the feasibility of lossless global distance matrix reconstruction and privacy preservation. Theorem 1 proves that when the number of clients satisfies $ m \\geq 2(l + t - 1) + 1 $, the distance matrix can be accurately reconstructed using Lagrange Coded Computing (LCC) and polynomial interpolation. Theorem 2 employs mutual information analysis $( I(\\hat{x}_i; \\{z_{i,j}\\}_{j \\in \\mathcal{C}}) = 0 ), ( |\\mathcal{C}| \\leq t )$ to ensure data privacy against collusion of up to $t$ clients. These proofs leverage finite field operations for numerical stability, assuming data can be segmented into $l$ parts with $t$ added noises. Experimental validation shows minimal reconstruction error (RMSE=0.0002), outperforming SOTA methods like k-FED and FedSC in non-IID scenarios, demonstrating superior robustness and generality.\n\n2.**Novelty of LCC.** Lagrange Coded Computing (LCC), as a distributed secure computing technique, can reduce computation and communication costs while ensuring fault tolerance and security. The authors ingeniously apply it to encode local client data for interactively computing the global clustering distance, which I consider quite novel in the field of privacy-preserving clustering.\n\n3.**Experiments.** OmniFC demonstrates superior performance across multiple datasets (Iris, MNIST, COIL-100, etc.) and five non-IID scenarios (p=0 to 1), with Kappa remaining nearly unchanged as non-IID intensity increases, outperforming SOTA methods like k-FED (0.88 to 0.62) and FedSC (0.90 to 0.65). Its model-agnostic design and global distance matrix reconstruction ensure high robustness and consistency, supporting various clustering algorithms.\n\nWeaknesses\n1.**Lack of Comparison Methods.** The paper presents a seemingly novel aggregation method, but it lacks citations or comparisons with recent works on secure or privacy-preserving federated aggregation. I suggest the authors at least reference and discuss the following works: \n1) Secure Federated Clustering, 2022, IACR; \n2) Differentially Private Federated k-Means Clustering with Server-Side Data, 2025, ICML; \n3) FastLloyd: Federated, Accurate, Secure, and Tunable k-Means Clustering with Differential Privacy, 2025, arXiv.\n\n2.**Insufficient Sensitivity Analysis.** The experiments use a single finite field scaling parameter $q$, without examining its impact on clustering performance. I recommend adding an ablation study on bit-width $q$, such as gradually reducing $q$, plotting the relationship between reconstruction error and clustering metrics Kappa & NMI, and discussing the impact of precision degradation.\n\n3.**Lack of Time and Computational Complexity Analysis.** The paper does not quantify the runtime or resource consumption of OmniFC. Secure coded computing often introduces overhead from polynomial interpolation. Without clarification, its feasibility is unclear. I suggest providing time complexity estimates for clients and servers, detailing computational costs (e.g., polynomial encoding or decoding, per-round distance computation), and discussing whether the overhead is acceptable in federated settings. This allows comparison with other methods to highlight efficiency advantages.\n\n4.**Limited Novelty.** The method appears similar to Secure Federated Clustering. Could the authors discuss the differences? LCC is not new in federated settings, and directly encoding data to estimate distances is not novel in federated learning, as seen in: \n1) DReS-FL: Dropout-Resilient Secure Federated Learning for Non-IID Clients via Secret Data Sharing, 2022, NIPS; \n2) Coded Computing for Low-Latency Federated Learning over Wireless Edge Networks, 2020, IEEE JSAC Series on Machine Learning for Communications and Networks; \n3) Lagrange Coded Federated Learning (L-CoFL) Model for Internet of Vehicles, 2022, ICDCS; \n4) FedWDP: A Wasserstein-distance-based federated learning for privacy and heterogeneous data in IoT, 2025, Internet of Things. \nThe paper seems to apply this approach to federated clustering (FC). Please re-emphasize the specific innovations.\n\n5.**Poor Exposition.** The concept of \"Federated Clustering (FC)\" is confusing. The authors fail to clearly define the application scenarios and task objectives in the introduction or background, leading to doubts about the goals of OmniFC.\n\n6.**Simple Non-IID Partitioning.** The authors use a single label skew parameter p to control non-IID settings. I suggest adopting more common partitioning strategies. Federated learning studies often use Dirichlet distributions with varying concentrations $\\alpha$ to model label heterogeneity (e.g., Khosravan et al., Redefining non-IID Data in Federated Learning for Computer Vision Tasks: Migrating from Labels to Embeddings for Task-Specific Data Distributions, 2025, arXiv) or feature shifts in image tasks. I recommend using Dirichlet($\\alpha$) for label allocation, observing performance changes as $\\alpha$ decreases (IID to highly non-IID). This would better validate robustness in real-world cross-device settings. Current results under a single skew are insufficient to demonstrate applicability across diverse non-IID scenarios.\n\nQUESTIONS:\nPlease refer to the weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addressed two critical challenges in federated learning: (1) privacy leakage during collaboration, and (2) robustness degradation due to the non-IID local data. This work introduces OmniFC, a unified and model-agnostic framework via lossless and secure distance reconstruction. In fact, the authors design a pipeline that employs Lagrange coded computing to generate intermediate exchange variables, which are then used to reconstruct the exact pairwise distance matrix. Therefore, their proposed approach remains broadly applicable to any machine learning tasks that rely on the pairwise distance matrix such as spectral clustering, SVM, and t-SNE. In this sense, the method is claimed to be model-agnostic. Additionally, the authors demostrate that OmniFC preserves data privacy, arguing that there is zero mutual information between the raw data and the intermediate exchange variables, which is a distinct idea from the traditional standard procedure such as differential privacy (DP), homomorphic encryption (HE), and secure multi-party computation (SMC). Theoretical and empirical results have shown that this framework improves robustness under non-IID settings.\n\nSTRENGTHS AND WEAKNESSES:\n__Strength__\n\nThis paper proposes the OmniFC framework, which uses lossless and secure distance reconstruction to address two common challenges in federated learning: privacy leakage during collaborative training and robustness degradation under non-IID settings. According to the existing literature, it is non-trivial to achieve a high-equality approximation of pairwise distance matrix in federated environments. OmniFC employs Lagrange encoded computing to reconstruct the EXACT pairwise distance matrix in a federated setting, which represents a significant and meaningful advancement for this problem. This contribution could benefit a wide range of machine learning tasks that heavily rely on accurate pairwise distance matrix reconstruction in federated scenarios. One limitation, however, is that the authors justify the privacy guarantee theoretically by claiming that there is zero mutual information between raw data and the intermediate exchange variables. I am not certain that mutual information alone provides a sufficient theoretical foundation for privacy preservation, as I am not an expert in encryption. Nevertheless, this work is interesting and the paper is concise and well-written.\n\n__Weakness__\n\nI have some question about this work as follows:\n\n1. Lines 81-87: It would be clearer to remove the description of the proposed framework from the Related Work section and focus this section solely on reviewing relevant prior studies.\n\n2. The expression $f_{\\tilde{x} _i,\\tilde{x} _{i'}}$ is not defined in the statement of Theorem 1. Since it is explained in the appendix, it may improve clarity to directly specify its definition within Theorem 1.\n\n3. In Theorem 1, it is claimed that the distance $\\text{dis}(\\tilde{x} _i,\\tilde{x} _{i'})$ can be precisely recovered by $\\sum _{o=1}^l f _{\\tilde{x} _i,\\tilde{x} _{i'}}(\\alpha _o) = \\sum _{o=1}^l \\|f _{\\tilde{x} _i}(\\alpha _o) - f _{\\tilde{x} _{i'}}(\\alpha _o)\\| _2^2 = \\sum _{o=1}^l \\|s _{i,o} - s _{i',o}\\| _2^2$. However, it is $z _{i,j} = f _{\\tilde{x} _i}(\\beta _j)$ as the transmitted value instead of $f _{\\tilde{x} _i}(\\alpha _o)$. Furthermore, since $\\{\\alpha _o\\} _{o=1}^{l+t}\\cap\\{\\beta _j\\} _{j=1}^m = \\phi$, it appears that the server has no access to $f _{\\tilde{x} _i}(\\alpha _o)$ for $i\\in[n]$ and $o\\in[l]$. Could the authors clarify how the server can recover $\\text{dist}(\\tilde{x} _i,\\tilde{x} _{i'})$ in practice from the uploaded set $\\{\\text{dis}(z _{i,j}, z _{i',j})\\} _{j=1}^m$? It would be helpful if the authors could provide a brief proof or explanation of this recovery process.\n\n4. Line 159: How can $X$ be obtained from the recovered proxy $D$ for clustering methods that do not explicitly depend on pairwise relationships, as considered in this work?\n\n5. In the case of two clients ($m=2$), will the proposed method remain effective? In other words, how should $l$ and $t$ be selected to ensure that the condition $m > 2l + 2t - 1$ holds given $m = 2$?\n\n6. Why does the severity of the non-IID setting not significantly affect the performance of the KM-based, FCM-based, and NMF-based methods? Furthermore, why does FFCM perform better under severe non-IID scenarios, even on the 10x\\_73k dataset?\n\n7. The numerical sensitivity results indicate that the proposed method successfully reconstructs the distance matrix when the condition $m > 2l + 2t - 1$ holds. It would be helpful if the authors could also provide the reconstruction loss for scenarios where this condition does not hold?\n\nQUESTIONS:\nPlease see the weakness.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes **OmniFC**, a novel federated clustering (FC) framework that addresses privacy leakage and robustness degradation under non-IID data via lossless, secure reconstruction of the global distance matrix. Existing FC methods rely on model-specific proxies (e.g., centroids or low-rank factors), which are sensitive to data heterogeneity and inherit biases from centralized clustering. OmniFC leverages **Lagrange coded computing (LCC)** to enable clients to share encoded data, allowing exact reconstruction of pairwise distances without raw data exposure—even under client collusion. The framework is model-agnostic, unifying extensions for diverse centralized clustering methods (k-means, spectral clustering, DBSCAN, etc.). Theoretical guarantees for reconstruction fidelity and privacy are provided, and experiments across 7 benchmarks show superior robustness and effectiveness over SOTA methods.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**   \n1. **Groundbreaking Privacy-Security Integration**: First to achieve *lossless* distance matrix reconstruction under LCC encryption, providing information-theoretic privacy against colluding clients (Theorem 2). This resolves a core FC challenge.  \n2. **Unified Model-Agnostic Design**: Decouples clustering from model-specific proxies. The framework’s generality is a significant advancement.  \n3. **Theoretical Rigor**: Theorems 1–2 formally guarantee exact distance reconstruction (under \\(m \\geq 2l + 2t - 1\\)) and \\(t\\)-private security, with tight alignment between efficacy and privacy conditions.  \n4. **Compelling Empirical Results**: Extensive experiments on tabular, image, and genomic data.\n5. **Exceptional Clarity**: Well-structured with intuitive figures (Fig. 2–3) and detailed pseudocode (Appendix A).  \n\n\n**Weaknesses:**  \n1. **Scalability Limits**: Although the paper proposes segmentation and encoding techniques to reduce communication costs, computing and reconstructing the global pairwise distance matrix (of size $n \\times n$) can be computationally and memory-intensive, especially on large-scale datasets such as 10x_73k (with 73k samples). While the authors employ a subsampling strategy (1,000 samples) to alleviate this issue, the paper does not adequately discuss how the method can be scaled to larger datasets in practical federated learning scenarios.\n\n2. **Hyperparameter Sensitivity**: Parameters \\(l\\) (segments), \\(t\\) (noises), and \\(m\\) (clients) must satisfy \\(m \\geq 2l + 2t - 1\\). Tuning guidance is lacking, and violating this condition causes reconstruction failure (Fig. 5).  \n\n3. **Shallow Clustering Focus**: Validated only on classical methods (k-means, SC, etc.). Extension to deep clustering (Appendix E) is speculative and untested.\n\nQUESTIONS:\nPlease check it in Weekness.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}