# Research into Robust Federated Learning Methods Driven by Heterogeneity Awareness
## 2 Preliminary Knowledge  
### Federated Learning  
Federated learning (FL) is a distributed machine learning paradigm that facilitates collaborative model training across multiple clients while preserving data privacy, as depicted in Figure 1. In practice, however, client data are typically non-independent and identically distributed (non-IID), resulting in divergent local updates and a degraded global performance. This heterogeneity poses significant challenges to models' convergence and generalization.  
To mitigate the impact of data heterogeneity, a wide range of optimization techniques have been proposed, which can be broadly categorized into three groups: model-level alignment, loss function compensation, and aggregation strategy enhancement.  
To reduce local model drift, FedProx introduces an L2-based proximal term that constrains local updates [22]. Federated Curvature Regularization (FedCurv) employs the Fisher information matrix to guide regularization along important parameter directions [23]. Similarly, federated sharpness-aware minimization (FedSAM) improves robustness by incorporating neighborhood and global perturbations into a sharpness-aware minimization,  
Figure 1: Federated learning training process.  
making the model more globally aware while maintaining local consistency [24]. Federated Aggregation-Free (FedAF) further reduces client drift by allowing clients to collaboratively learn using condensed data and soft labels, addressing data heterogeneity and improving global model performance while avoiding the problems caused by traditional aggregation [25]. Although these approaches improve the robustness to some extent, they typically lack an explicit modeling or quantification of inter-client heterogeneity.  
Several recent works focus on aligning client representations rather than model parameters. Federated Representation Learning (FedRep) decouples the model into a shared feature extractor and a personalized classifier to enhance its generalizability. Federated Model Aggregation (FedMA) performs layer-wise matching to aggregate models with heterogeneous architectures. FedMOON incorporates contrastive learning to ensure local representation consistency across federated rounds, reinforcing client-specific feature learning [26; 27; 28]. Personalized Federated Learning with Model-Contrastive Learning (PFL-MCL) enhances multi-modal user modeling in Metaverse environments, addressing data and model heterogeneity to improve communication efficiency and networking in human-centric applications [29]. However, most of these methods do not directly measure or model data heterogeneity, limiting their theoretical tractability and optimization adaptability.  
Some studies attempt to explicitly account for heterogeneity during aggregation. FedNova addresses the bias introduced by unequal local update steps via normalized aggregation [30]. On the other hand, Federated Domain Shift Eraser (FDSE) mitigates domain shifts in federated learning by deskewing features with a Domain-agnostic Feature Extractor (DFE) and Domain-specific Skew Eraser (DSE), enhancing models' generalization across diverse client distributions [31]. Additionally, Hierarchical Split Federated Learning (HSFL) optimizes federated learning by splitting models and aggregating them efficiently, enhancing performance under multi-tier systems, especially for resource-constrained edge devices [32]. Nevertheless, these methods largely operate at the aggregation level and do not capture feature-level distributional discrepancies.  
To address these limitations, this work proposes a differentiable heterogeneity measurement approach based on the statistical characterization of client feature distributions. This heterogeneity metric is integrated with a multi-loss optimization framework and a heterogeneity-aware aggregation strategy, aiming to enhance models' adaptability and stability in non-IID environments while ensuring interpretability and generalization.