## 2 Related Work and Motivation
### Federated Learning  
_Federated Learning (FL)_[5; 7; 13] is a decentralized machine learning paradigm where multiple clients (such as edge devices or distributed nodes) collaborate to train a shared model without exposing their raw data. FL is particularly well-suited for scenarios involving sensitive or distributed data, such as healthcare, finance, or network security [14]. FedAvg [5] is a FL foundation algorithm where clients perform local updates and send model parameters to a central server for aggregation. While it reduces communication cost and supports scalability, FedAvg suffers under non-Independent and Identically Distributed (non-IID) data distributions [7; 13; 14], where client data varies due to user behavior, geographic location, or deployment context, which can significantly hinder the convergence of the global model and degrade global model performance.  
To address the non-IID data distribution challenges in FL, many studies have proposed _clustering-based solutions_. Instead of aggregating model updates from all clients regardless of their data characteristics, the core idea is _grouping clients with similar data distributions_, which can reduce gradient conflicts, stabilize training, and improve convergence [8; 9; 15]. In essence, clustering acts as a regularization mechanism that mitigates the divergence caused by data heterogeneity across multiple client devices.  
Based on the idea and effectiveness of clustering clients, several studies have been conducted. [PERSON] et al. [8] developed WSCC, a clustering method based on weight similarity, which groups clients according to the similarity of their model characteristics after local training. This approach enhances stability under non-IID data conditions without requiring access to individual client data. This enhances learning stability under non-IID settings without requiring access to client data. K-FL [15] leverages a Kalman filter to dynamically estimate client similarity, reducing computation time and training variance. FedPVD [9] improves robustness by clustering clients according to the directional changes in their model parameters across rounds. IHC-FL [10] enhances performance by enabling intra-cluster update exchanges, thereby minimizing global communication overhead and local overfitting. [PERSON] et al. proposed FLIS [11], a clustered FL approach designed to mitigate the impact of non-IID data by grouping clients based on the similarity of their model outputs on a shared auxiliary dataset, allowing it to more accurately reflect the underlying data distribution.  
Table 1 presents a comparison of FL methods that use client clustering techniques. Among them, weight-based approaches (WSCC, K-FL) are simple but require transmission of large models. FLIS effectively addresses strong non-IID scenarios but demands a shared auxiliary dataset, which conflicts with privacy objectives. FedSAGE, on the other hand, only transmits small hidden vectors and requires neither labels nor auxiliary data, making it lightweight in communication and better at protecting raw data.  
Despite these advances, clustering based on model parameters has notable limitations. Model parameters do not always accurately reflect the underlying characteristics of the input data [16], especially when local training is minimal. To improve clustering precision, such approaches typically require multiple rounds of local training per client, increasing computation costs. Furthermore, transmitting large model parameters for clustering purposes adds significant communication overhead [12], which is especially problematic when working with large-scale models. Therefore, to develop a stable FL system under such complex conditions, it is essential to design a client clustering method that captures the underlying data distribution characteristics while ensuring strong privacy protection, high accuracy, and low communication cost.