# A Review and Experimental Evaluation on Split Learning
## 3 Experimental Evaluation
### Experimental Results and Analyses
#### 3.2.2 Comparison in the Non-IID Setting
As shown in Figure 7, the performance of SplitFed remains stable when the data are divided by the Dirichlet distribution with a small \(\alpha\) (e.g., \(\alpha=0.1\)), achieving an accuracy of 97.68%, which is close to the accuracy achieved on centralized training. However, the accuracy ratios of Vanilla and PSL are low when the value of \(\alpha\) is small. We also observe similar trends when training on the CIFAR-10 dataset. This may be because SplitFed allows the clients to send their model parameters to the server, which facilitates the exchange of information about local data. Unlike SplitFed, the clients in SGLR share the information by only averaging the local gradients of the cut layer. As a result, for small values of \(\alpha\), it outperforms other paradigms except for SplitFed. For example, SGLR achieved an accuracy of 67.65% when \(\alpha=0.1\), but Vanilla has an accuracy of only 34.24% in the same setting. Figure 6: Illustration of data partitioning based on the Dirichlet distribution. In the first row, the number of clients is set to 3; and in the second row, the number of clients is set to 5. * Vanilla performs worse than other paradigms and even worse than local training in some cases. As shown in Table 5, Vanilla only achieves a 18.26% accuracy when \(\alpha=0\). The training mode of Vanilla prevents the model from converging in many cases. * In addition, we observe that the client number can also affect the model performance. Specifically, in Tables 6 and 7, when the number of clients is 5 and \(\alpha\) is small, most SL paradigms achieve a much lower accuracy than when the number of clients is 3. This may be because the data become more heterogeneous as the number of clients increases. As an example, a client may have samples from 3 or 4 classes when the number of clients is 3, but it can only have samples from 2 classes when the number of clients is 5. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Paradigm**} & **IID** & \multicolumn{4}{c}{**Dirichlet Distribution**} \\ \cline{3-6}  & **(\(\alpha=\infty\))** & \(\alpha=0\) & \(\alpha=0.05\) & \(\alpha=0.1\) & \(\alpha=0.3\) \\ \hline Vanilla & 97.92\% & 18.26\% & 69.37\% & 72.88\% & 90.24\% \\ U-shaped & 97.13\% & 32.97\% & 60.24\% & 70.19\% & 93.78\% \\ SplitFed & 98.82\% & 79.32\% & 96.87\% & 97.68\% & 97.39\% \\ PSL & 97.28\% & 34.86\% & 50.54\% & 67.64\% & 85.00\% \\ SGLR & 97.34\% & 33.12\% & 60.71\% & 86.03\% & 94.79\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Accuracy of SL paradigms on the MNIST dataset after 30 epochs with 3 clients [108]. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Paradigm**} & **IID** & \multicolumn{4}{c}{**Dirichlet Distribution**} \\ \cline{3-6}  & **(\(\alpha=\infty\))** & \(\alpha=0\) & \(\alpha=0.05\) & \(\alpha=0.1\) & \(\alpha=0.3\) \\ \hline Vanilla & 98.17\% & 10.46\% & 10.31\% & 12.53\% & 93.87\% \\ U-shaped & 97.46\% & 19.12\% & 10.41\% & 10.84\% & 92.36\% \\ SplitFed & 98.76\% & 90.13\% & 95.95\% & 96.67\% & 98.39\% \\ PSL & 98.37\% & 19.86\% & 70.63\% & 78.63\% & 95.22\% \\ SGLR & 98.26\% & 10.20\% & 56.91\% & 86.03\% & 90.87\% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Accuracy of SL paradigms on the MNIST dataset after 30 epochs with 5 clients [108]. Figure 7: Performance of four SL paradigms on different \(\alpha\)s when the number of clients is 3 on the MNIST and CIFAR-10 datasets. Here, the x-axis represents the number of epochs. Also, non-IID data may exacerbate the backward client decoupling problem [58], which is that the models trained on different clients' data may have significant discrepancies. For example, in PSL, the accuracy of different clients becomes increasingly different as the heterogeneity of the data increases, as shown in Figure 8. This is because in PSL, the clients only use their local data's gradients for training, and the clients become decoupled during the training.