## 5 Simulation
### Simulation Results  
Assume that \(\beta=1\) and \(\tau=20\) s. Figure 5 shows the test accuracy of the three algorithms over the global iterations and simulation time for CIFAR-10 and MNIST. From Figure 5a,b, we can find that LESSON and FedAvg have similar test accuracy, i.e., \(\sim\)70% for CIFAR-10 and \(\sim\)95% for MNIST. However, the test accuracy achieved by FedCS is lower than LESSON and FedAvg, i.e., \(\sim\)60% for CIFAR-10 and \(\sim\)90% for MNIST. This is because FedCS only selects fast clients to participate in the training process, and so the derived global model can only fit the data samples for fast clients, not slow clients, thus reducing the model accuracy. Meanwhile, the convergence rate with respect to the number of global iterations for LESSON and FedAvg is also very similar, which is slightly faster than FedCS. However, by evaluating the convergence rate with respect to the time, as shown in Figure 5c,d, we find out that LESSON is faster than FedAvg. For example, the  
Figure 4: Probability distribution of 10 categories samples for 5 clients with different \(\beta\). global model in LESSON has already converged at 20,000 s for MNIST, but the global model in FedAvg is still under-trained. This is because the FL server in FedAvg must wait until the local models from all the clients have been received in each global iteration, and thus, the latency of a global iteration incurred by FedAvg is much higher than that incurred by LESSON. Table 2 shows the average delay of a global iteration incurred by different algorithms, where the average latency of a global iteration incurred by LESSON is 48 s faster than FedAvg. As a result, FedAvg only runs around 588 global iterations at 20,000 s in Figure 5d, respectively, but LESSON runs 2000 global iterations. We further investigate how the data sample distribution affects the performance of the algorithms based on CIFAR-10. As mentioned before, \(\beta\) is used to change the data sample distribution, i.e., a larger \(\beta\) implies a more balanced data partition among the labels in a client or data sample distribution much closer to IID, and vice versa. Assume \(\tau=20\) seconds, and Figure 6 shows the test accuracy of different algorithms over the number of global iterations by selecting different values of \(\beta\). From the figures, we can see that if the data sample distribution exhibits non-IID, i.e., \(\beta=0.1\), FedAvg has higher test accuracy than LESSON and FedCS. As \(\beta\) increases, i.e., the data sample distribution is growing closer to IID, the test accuracy gap between FedAvg and LESSON is growing smaller, while the test accuracy of FedCS remains unchanged. On the other hand, Figure 7 shows the test accuracy of different algorithms over time by selecting different values of \(\beta\). From the figure, we can see that LESSON achieves 2\(\times\) faster convergence rate than FedAvg under different values of \(\beta\). Therefore, we conclude that LESSON achieves a faster convergence rate at the cost of slightly reducing the model accuracy, especially when the data distribution exhibits non-IID. \begin{table}
\begin{tabular}{c c} \hline \hline
**Algorithms** & **Average Latency of a Global Iteration** \\ \hline FedAvg & 68 s \\ FedCS & 20 s \\ LESSON & 20 s \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Average latency per global iteration for different algorithms. Figure 5: Test accuracy of different algorithms for CIFAR-10 and MNIST with \(\beta=1\), where (**a**) test accuracy vs. the number of global iterations for CIFAR-10, (**b**) test accuracy vs. the number of global iterations for MNIST, (**c**) test accuracy vs. time in CIFAR-10, and (**d**) test accuracy vs. time in MNIST. Figure 6: Test accuracy over the number of global iterations for CIFAR-10, where (**a**) \(\beta=0.1\), (**b**) \(\beta=1.0\), and (**c**) \(\beta=10.0\). As mentioned in Section 4.3, the deadline of a global iteration, i.e., \(\tau\), is a very important parameter to adjust the performance of LESSON.