# Advanced Optimization Techniques for Federated Learning on Non-IID Data
## 3 Methodology  
### Tools and Datasets  
In this study, the implementation of the federated learning system was supported by a set of cutting-edge tools, selected for their efficiency, flexibility, and broad community support. These tools include Google Colab for code development and execution, TensorFlow for building and training machine learning models, TensorFlow Federated (TFF) for implementing federated learning, and Apache Spark (PySpark) for data pre-processing and partitioning. * **Google Colab:** A cloud-based development environment provided by Google, designed for machine learning and data science research. It is based on Jupyter Notebook and provides free access to CPU and GPU resources, allowing users to write and execute Python code through their browser [45]. * **TensorFlow:** An open-source software library developed by Google for numerical computation and machine learning. It facilitates the creation and training of neural networks, offering a wide array of libraries and tools, making it a leading tool in artificial intelligence and scientific research. * **TensorFlow Federated (TFF):** An open-source framework that extends TensorFlow, specifically designed for federated learning. TFF allows machine learning models to be trained on decentralized data while ensuring data privacy and security. It includes two main APIs: the Federated Core (FC) API for low-level distributed computation and the Federated Learning (FL) API for high-level federated training and evaluation [46; 47]. * **Apache Spark (PySpark):** Apache Spark is an open-source framework that serves as a powerful tool for processing and analyzing large datasets in distributed environments. Spark supports multiple programming models and APIs, including the Resilient Distributed Dataset (RDD) and DataFrame API, allowing users to perform data processing and analysis tasks in a simple and optimized manner. PySpark is the Python API for Spark, enabling users to leverage all the capabilities of Spark through the Python programming language [48]. The selection of datasets for training and evaluating models in federated learning is crucial for understanding the behavior and performance of these models in environments with highly non-IID data. For the implementation of our experiments, three datasets were used: MNIST, Fashion MNIST, and CIFAR-10. These datasets were chosen to demonstrate the impact of IID and non-IID data distribution in federated learning environments and to address the challenges posed by non-IID data effectively. * **MNIST:** The MNIST (Modified National Institute of Standards and Technology) dataset is a comprehensive collection of handwritten digits, consisting of a training set of 60,000 examples and a test set of 10,000 examples. The dataset includes grayscale images of handwritten digits, each of size 28 \(\times\) 28 pixels, normalized and centered in a 28 \(\times\) 28 grid. MNIST is widely used for training and testing machine learning algorithms, particularly for image classification tasks using Convolutional Neural Networks (CNNs), Support Vector Machines (SVMs), and other machine learning algorithms. Its simple and well-organized structure makes MNIST a foundational tool for researchers in machine learning and computer vision [49]. * **Fashion MNIST:** The Fashion MNIST dataset is a modern and more challenging alternative to the traditional MNIST, consisting of images representing various clothing items. It contains 70,000 grayscale images, each 28 \(\times\) 28 pixels, categorized into 10 classes, with 60,000 for training and 10,000 for testing. Similar to MNIST, the dataset includes fields for the image and corresponding label. The diversity of clothing items, coupled with their similarities, introduces a greater level of complexity, testing the generalization capabilities of machine learning models [50].