Poster: Improved Federated Learning with Non-IID Data Using Foundation Models Fatima Abacha∗, Sin G. Teo†, Lucas C. Cordeiro∗and Mustafa A. Mustafa∗‡ ∗Department of Computer Science, The University of Manchester, Manchester, M13 9PL, UK †Institute for Infocomm Research, A*STAR Singapore ‡COSIC, KU Leuven, Leuven, 3001, Belgium Email: fatima.abacha@postgrad.manchester.ac.uk Abstract—Federated Learning (FL) enables multiple parties to train a model without sharing data. However, in heteroge- neous scenarios where the data distribution amongst the FL participants is non-independent and identically distributed (non-IID), FL suffers from the data heterogeneity challenge which severely degrades the ability of the global model to converge. To solve this problem, we propose a novel data augmentation strategy, named DPSDA-FL, which can aid in homogenizing the local data present on the client’s side. DPSDA-FL improves the training of the global model by leveraging differentially private synthetic data from founda- tion models. We obtain promising preliminary results on the CIFAR-10 dataset regarding recall of the global model. 1. Introduction Federated Learning (FL) enables multiple parties to come together and train an ML model without sharing their data [1]. The training process is orchestrated by a third party, which is usually a central server. In FL, each client uses private data to train its own local model, while the server uses an aggregation algorithm to construct a global model. The entire process runs for several iterations until a global model with the desired performance is achieved.This global model is then broadcast to all clients so they can use it for inference on their test dataset. FL protects against data leakage as each client’s private training data is not disclosed to any other party. FL also enables adherence to regulations such as the GDPR [2]. However, when the data distribution amongst the clients in FL is statistically heterogeneous, meaning the data distribution is non-independent and identically dis- tributed (non-IID), the prediction accuracy of the models is affected. A client may hold data from some classes and not from other classes present in the global dataset, as such the ability of the global model to make accurate inferences is severely degraded [3]. Also, when clients train their local model on data that does not contain certain classes from the global set or only a few samples from specific classes, the models will likely be biased towards those underrepresented groups [3]. This could have devastating consequences when these models are deployed in safety- critical situations such as healthcare and finance. Statistical heterogeneity can be tackled by making the dataset across FL clients uniform in their distribution. This can be achieved by using data augmentation, a technique that can generate more training data to harmonize the data distribution amongst the clients in FL. Data augmentation is effective as it can reduce the problem of non-IID data in FL [3]. Existing works [4], [5] in the literature have used Generative Adversarial networks (GANs) to generate synthetic data for data augmentation. GANs, however, are vulnerable and tricky to train to produce high-quality and diverse synthetic data for data augmentation [6]. In this regard, we propose using foundation models for a more effective data augmentation process. To the best of our knowledge, this is the first work that employs pre-trained foundation models to generate differentially private synthetic data to tackle the problem of non-IID Data in FL. Thus, our contributions are as follows: • We propose a new data augmentation strategy, named Differentially Private Synthetic Data Aided Federated Learning using Foundation Models (DPSDA-FL), to enhance the FL performance with non-IID Data. • We demonstrate the effectiveness of utilizing Dif- ferentially Private Synthetic Data from Foundation Models in Cross Silo Horizontal FL. • We conduct exp