## 4 Results and Discussion
### Discussion
Two-level aggregation [164; 175] is where a middle layer near the clients aggregates the model to reduce the communication rounds with the central server. Even though most of the techniques are based on reducing the number of updates, most of the selected studies employ techniques to reduce the communication bits such as quantization and sparsification, while a few studies aimed to reduce the number of bits and the number of rounds in their approaches be employing different techniques such as quantization with client selection [191] and with periodic model averaging [198]; as shown in Figure 22, 61% of the selected studies apply a technique that aims to reduce the communication bits. Figure 16 shows the top techniques that are commonly used in studies that address both challenges (non-IID data challenge and providing efficient communication). Most studies used a technique for each challenge, while some studies used one technique to overcome both challenges. These techniques are clustering, two-level aggregation, and client selection. Clustering is the most widely used technique for overcoming both challenges, appearing in 16% of non-IID data studies, 4% of communication-efficient studies, and 11% of studies that provided solutions to both challenges. In the cluster technique, the central server mostly clusters the clients into different groups based on different criteria such that similar clients are grouped to enhance the learning process and reduce the communication rounds. Two-level aggregation also can help reduce the communication rounds between the central server and the clients by introducing a partial aggregation near the clients that can reduce the number of models transmitted and enhance the model training. Client selection is an essential step in the federated learning performed by the server; selecting  
Figure 21: Categories of techniques to provide efficient communication in federated learning. the clients based on their model or resources enhances the training process and reduces the communication rounds. To answer RQ4, we extracted the most commonly used learning models in the selected studies, as shown in Figures 9, 13 and 17. The figures demonstrate that most of the studies utilized CNN as their learning model; it was used in 56% of non-IID data studies, 42% of communication-efficient studies, and 65% of studies that provided solutions to both challenges. Different learning models such as ResNet and VGG were also widely utilized in these studies. These learning models are supervised learning models and are often employed in FL. To address RQ5, we examined the selected studies and retrieved the most commonly used datasets, as shown in Figures 10, 14 and 18. The figures demonstrate that most studies use image datasets; in particular, they utilize Cifar-10 and MNIST datasets to evaluate their work. The Cifar-10 dataset was used in 60% of non-IID data studies, 54% of communication-efficient studies, and 62% of studies that provided solutions to both challenges, while the MNIST dataset was used in 49% of non-IID data studies, 55% of communication-efficient studies, and 54% of studies that provided solutions to both challenges. These datasets are used as benchmarks in FL to simulate IID and non-IID data distribution. Many studies simulate this skew using MNIST and Cifar-10 datasets, as they can be easily understood and evaluated by others. Besides, FL is a relatively new approach, researchers focus on addressing the challenges in clear-cut scenarios, especially the non-IID data challenge, as the studies need to simulate a non-IID data distribution that can be easily understood and evaluated by others. Researchers can use commonly used datasets such as MNIST and Cifar-10 to gain valuable insights into the effectiveness of proposed methodologies and compare them with existing approaches. Furthermore, the first study that proposed FL utilized these datasets and the Shakespeare dataset in their work.