# A Fairness-Aware Peer-to-Peer Decentralized Learning Framework with Heterogeneous Devices
## 5 Performance Evaluation
### Experimental Results  
We first validate our approach and two baseline approaches in the balanced-i.i.d. data scenario. Figure 3 shows the accuracy distribution of all three approaches. As seen in the figure, all of the approaches achieve the acceptable performance where the learning performance of the majority of the participants is higher than 90%. This is because a balanced-i.i.d. data distribution scenario eliminates the diversity of local learners, which makes all of the local learners become homogeneous. Similar to Figure 3, we obtain the accuracy distribution of all of the approaches after we have 2000 local learning pairs on the CIFAR-10 dataset. As shown in Figure 4, the performance gap of our approach and two baseline approaches is not significant. On the other hand, over half of the local learners could achieve the acceptable performance which is between 60% to 75%. In Figures 5 and 6, we demonstrate the evaluation results in the unbalanced-i.i.d. data distribution scenario. Both of them show that all of the approaches could achieve a comparable learning performance in a homogeneous learning scenario. Compared to case 1 (balanced-i.i.d. data distribution) and case 2 (unbalanced-i.i.d. data distribution), we consider the decentralized learning processes in case 3 (balanced-non.i.i.d.) and case 4 (unbalanced-non.i.i.d.) for statistical heterogeneity. As shown in Figure 7, noticeably over half of the local learners reach 90% (local accuracy) on the E-MNIST dataset. Figure 8 shows that our approach achieves the best learning performance among all of the tested approaches where 15 nodes reach 55% (local accuracy) on the CIFAR-10 dataset. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Total node number & 20 \\ \hline Local epoch & 2 \\ \hline Non-i.i.d. coefficient \(\zeta\) & 2, 3, 4 \\ \hline Minibatch size & 100 \\ \hline Learning pair & 1000/2000 \\ \hline Total scenarios & 4 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: List of parameters. Additionally, we record the evaluation results of case 4 in Figures 9 and 10. Figure 9 shows that our approach performs the best among all of the tested approaches where 14 nodes reach 80% (local accuracy) on the E-MNIST dataset. When it comes to the CIFAR-10 dataset, our approach, as seen in Figure 10, shows a great capability, which not only successfully makes eight local learners to achieve the acceptable learning performance but also avoids resulting in local learners with an extremely low performance (30% to 45%). There are a number of reasons that lead to these evaluation results. We find that the decentralized learning with a statistical heterogeneous setting brings the inevitable challenge which degrades the learning performance for all of the tested approaches. To strengthen the local learning objectives, our approach aggregates the local learning model with the consideration of its learning performance. In case 4 (unbalanced-non.i.i.d. scenario), we assign two nodes with the massive local dataset (10,000 samples), resulting in these two nodes bringing negative impacts on other learning pairs. By using Equations (4) and (5), our proposed fairness-aware approach allows each node to train the local learning model with its own learning preference and objective. To be specific, in local aggregations, each local node evaluates the received learning model based on its local learning objective to determine the weight of aggregation. In this sense, the experimental results, in case 4 (unbalanced-non.i.i.d. scenario), clearly validate that our approach eliminates the negative impact of statistical heterogeneity (both the non-i.i.d. data distribution and unbalanced dataset) and that it outperforms the baseline approaches. Furthermore, we present numerical results in Tables 3-5, showing that our approach achieves the best performance under most cases (case1-E-MNIST, case3-E-MNIST, case4-E-MNIST, case2-CIFAR-10, case3-CIFAR-10, and case4-CIFAR-10). Figure 5: Accuracy Distribution for Case 2 (E-MNIST).