# Advanced Optimization Techniques for Federated Learning on Non-IID Data
## 6 Conclusions and Future Work
Compared to the CLR approach, the data sharing and pre-training on augmented data strategy allowed the model to achieve an evaluation accuracy of 51.4% in 200 training rounds, reflecting an increase in accuracy of 5.3%. Additionally, this strategy allows reaching the desired evaluation accuracy 2.72 times faster than the CLR approach, thus reducing the required communication rounds from 90 to 33. Based on our experimental results, we conclude that the implementation of the data sharing and pre-training strategy on augmented data combined with the cyclic learning rate policy is an effective solution to the important problem presented by non-IID data in federated training. This approach not only enhances the generalization capability of the model but also accelerates the speed of model convergence, facilitating the application of federated learning to real-world applications and reducing the time and resources required to train an efficient model. While this study focuses on improving federated learning performance across different datasets, there are further areas within federated learning that offer opportunities for research and development, such as reducing communication costs and enhancing protocol security. Communication between clients and the central server is one of the most costly processes in FL, leading to the development of data compression methods and improved communication protocols. For example, the study [63] presents FedZip, a compression strategy for federated learning that reduces communication costs by applying Top-z sparsification, quantization, and compression techniques, achieving high compression rates in model weight updates transmitted between clients and servers. Regarding security and privacy, ref. [64] proposes a method for implementing differential privacy in federated learning from the client side, aiming to protect client contributions while maintaining model performance. Furthermore, ref. [65] presents a secure aggregation protocol that ensures privacy in computing the sum of data vectors held by users without revealing individual contributions, addressing potential attacks, and improving communication efficiency even for large datasets. Future investigations will include implementing DeepSMOTE [66] into our federated learning framework because of its demonstrated success in managing class imbalances in centralized machine-learning scenarios. The research we are currently conducting emphasizes data augmentation methods outlined for federated learning, while further examination of DeepSMOTE as a potential solution for balancing classes in non-IID data could provide greater enhancements to model accuracy and convergence [66]. The approach taken during this exploration will enhance the privacy-preserving principles of federated learning. While our study demonstrates the effectiveness of Cyclical Learning Rate (CLR) and pre-training techniques on augmented data within a federated setup involving up to 1000 and 500 clients, further exploration is needed to investigate the scalability of these methods in environments with significantly larger client populations. Scaling to a larger number of clients is a critical area of exploration, as it reflects real-world applications where large-scale deployments are common. However, the current study was conducted within the limitations of datasets that were suitable for a smaller client pool. As a result, it may not fully capture the complexities encountered in larger-scale federated learning systems, where challenges such as increased communication overhead and data diversity play a more pronounced role. These limitations highlight the need for further investigations that utilize larger datasets and more extensive client populations to gain deeper insights into the scalability and generalizability of the proposed optimization techniques. Furthermore, for the purpose of our research, we decided to focus on an exaggerated case of strongly non-IID settings where each client had data from only two classes.