# Advanced Optimization Techniques for Federated Learning on Non-IID Data
## 6 Conclusions and Future Work  
This study focuses on analyzing and improving the performance of federated learning in environments characterized by highly non-IID data. Our experimental results led to several conclusions about the impact of non-IID data on FL performance and the improvements achieved through the strategies we implemented. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Training Rounds** & **SGD Fixed Lr** & **SGD Cyclical LR (CLR)** & **SGD CLR + PreTrain** \\ \hline  
\hline \end{tabular}
\end{table} 
Table 11: Comparative performance and percentage improvements. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Metric** & **Fixed** & **CLR & **PreTrain** & **\% Improv. (CLR over (CLR+PreTrain (CLR+PreTrain over Fixed))** & **\% Improv. (CLR+PreTrain over Fixed)** & **\% Improv. (CLR)+PreTrain over CLR)** \\ \hline
**Max** & 37.8 & 48.8 & 51.4 & 29.1\% & 36\% & 5.3\% \\
**Accuracy (\%)** & 176 & 90 & 33 & 1.95x & 5.33x & 2.72x \\ \hline
**Speedup (rounds)** & \(\mathrm{x1}\) & \(\mathrm{x1.95}\) & \(\mathrm{x5.33}\) & 95\% & 433\% & 173\% \\ \hline \hline \end{tabular}
\end{table} 
Table 10: Evaluation accuracy differences per 10 training rounds. Training models on three distinct datasets--MNIST, Fashion MNIST, and CIFAR-10--demonstrated that the effectiveness of federated learning significantly decreases when applied to non-IID data compared with IID data. For instance, in the case of the MNIST dataset, training on non-IID data led to a 3.09% decrease in accuracy and a 3.54-fold increase in the number of communication rounds required to achieve the target accuracy of 94%, compared with training on IID data. The impact was more pronounced for the Fashion MNIST dataset, with a 13.41% accuracy reduction and a 10.67-fold delay in reaching the target accuracy of 71%. Finally, for the CIFAR-10 dataset, the effect of non-IID data was even more drastic, with a 28.85% decrease in accuracy leading to a 4.09-fold increase in the number of communication rounds required to achieve the target accuracy of 37%. These results confirm the significant challenge posed by non-IID data in federated learning. The primary goal of our efforts was to address these challenges by implementing the Cyclical Learning Rate (CLR) strategy and the data sharing and pre-training on augmented data strategy. The application of the CLR strategy significantly improved the performance of the FedAvg algorithm on non-IID data compared with the conventional use of a fixed learning rate, as demonstrated by the experimental results across the three datasets: MNIST, Fashion MNIST, and CIFAR-10. Specifically, for the MNIST dataset, the use of CLR resulted in a 3.2% increase in evaluation accuracy and a 3% reduction in the required communication rounds to achieve the target evaluation accuracy of 94%, compared with using a fixed learning rate. In the case of the Fashion MNIST dataset, progress was even more evident, showing an increase in accuracy of 10%, while the communication rounds required to achieve the desired 71% accuracy were reduced by a factor of 2.32. For CIFAR-10, the CLR strategy provided the most significant improvement, increasing accuracy by 29%, from 37% to 48%, and reducing the communication rounds needed to achieve the target evaluation accuracy of 37% by 1.96 times. Recognizing the need for further improvement in federated learning in non-IID scenarios, particularly for more complex datasets like CIFAR-10, we developed the data sharing and pre-training on augmented data strategy. Applying this strategy to the CIFAR-10 dataset yielded remarkable results, with the final model's accuracy increasing by 36% compared to the FedAvg approach with a fixed learning rate. Moreover, the time required to achieve the target evaluation accuracy of 37% was reduced by 5.33 times, decreasing the number of communication rounds from 176 to just 33.