###### Abstract
KD transfers knowledge from a complex teacher model to a simpler student model using a weighted combination of loss terms, enabling the student to learn both from ground-truth labels and the teacher's broader predictive patterns. In FL, KD has been adapted to distill global knowledge into local models, improving convergence and robustness to non-IID data. FedMD [21], for instance, leverages a shared public dataset where clients compute and exchange logits, allowing the server to aggregate these into a consensus that guides local training through KD. Similarly, FedDF [22] employs a server-side proxy dataset to refine a global model by distilling predictions from client models, enhancing robustness across varied architectures. Yet, the reliance on such proxy datasets poses a limitation, as curating a suitable dataset may not be feasible in all scenarios, particularly in resource-constrained or privacy-sensitive applications where server-side data availability is restricted. These advancements underscore the potential of KD in FL while highlighting the need for methods that can operate effectively without depending on external datasets. Recent works have sidestepped the need for curated external data. For instance, [PERSON] and [PERSON] [23] proposed KD-PDFL. Unlike FedMD or FedDF, KD-PDFL operates without any proxy dataset; instead, each client leverages its local private dataset to compute intermediate outputs, using these to measure model similarity via KD in a peer-to-peer fashion. [PERSON] et al. [24] introduced Federated Distillation (FD), which reduces communication costs by sharing model outputs instead of parameters, though it sacrifices accuracy in highly non-IID scenarios [25]. [PERSON] et al. [25] introduced a data-free KD method for FL: the server employs a lightweight generator to synthesize ensemble knowledge from client models, which is then broadcast to guide local training via KD. FedMKD, proposed by [PERSON] et al. [26], leverages Mutual Knowledge Distillation (MKD) alongside Elastic Weight Consolidation (EWC) to mitigate catastrophic forgetting in global models, achieving superior performance in non-IID scenarios and reducing communication overhead through uniform and exponential quantization techniques. PFedSKD, introduced by [PERSON] et al. [27], employs self-knowledge distillation to retain personalized knowledge, addressing unreliable teacher models by stratifying neural network layers and incorporating auxiliary classifiers. [PERSON] et al. [28] presented FedGKD, which regularizes local training with an ensemble of historical global models via KD, achieving strong performance but relying on increased communication. These studies underscore the synergy of KD and FL in tackling data heterogeneity, enhancing model robustness, and optimizing resource-constrained environments like edge computing, paving the way for more scalable and privacy-preserving machine learning frameworks. We propose a novel FL algorithm that integrates KD with a weighted combination loss to tackle non-IID challenges in a centralized setting. Our approach draws from the insight of bi-level optimization, separating global and local optimization and using the global model as a teacher to guide local models via KD, while optimizing both levels concurrently. Compared to methods such as FedDF or FedMD, which depend on proxy or public datasets, our algorithm operates solely on clients' private datasets, ensuring privacy preservation without external data dependencies. Unlike FD, which prioritizes communication efficiency, or KD-PDFL, which emphasizes personalization, our method balances global convergence and local regularization. Compared to FedGKD, we streamline the process by focusing on the current global model rather than historical ensembles, enhancing efficiency and robustness. We evaluate our algorithm on the MNIST dataset and a synthetic dataset under non-IID conditions, demonstrating improved accuracy and convergence speed over other state-of-the-art (SOTA) algorithms.