# A Review and Experimental Evaluation on Split Learning
## 3 Experimental Evaluation
### Experimental Results and Analyses
#### 3.2.5 Comparison with On-Device Local Training  
One of the primary objectives of SL is to enhance the model's performance for each client without requiring the exchange of raw data. Therefore, we compare the performance of SL with on-device local training. In these experiments, each client follows the same data partitioning process as in the other SL experiments and uses an incomplete dataset to train their local model without any communication.  
Our main findings are summarized as follows:  
* The results of local training and two SL paradigms on IID data with different numbers of clients are presented in the first row of Figure 11. On the MNIST dataset, there is a slight difference among the three algorithms, with only a small drop in local training accuracy at 10 clients. However, on the CIFAR-10 dataset, local training shows a significant drop, with an accuracy of only 71.47% at 10 clients, which is probably due to the complexity of the learning task. Unlike handwritten characters, CIFAR-10  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Paradigm**} & \multicolumn{2}{c}{**Small (MB)**} & \multicolumn{2}{c}{**Large (MB)**} \\ \cline{2-5}  & **Send** & **Receive** & **Send** & **Receive** \\ \hline SplitFed & 15.17 & 3120.24 & 17.09 & 3120.24 \\ PSL & 15.17 & 3120.24 & 18.14 & 3120.24 \\ AsyncSL & 8.38 & 532.12 & 9.23 & 563.12 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Communication cost per epoch on the CIFAR-10 dataset when the number of clients is 3.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Paradigm**} & \multicolumn{2}{c}{**Small (MB)**} & \multicolumn{2}{c}{**Large (MB)**} \\ \cline{2-5}  & **Send** & **Receive** & **Send** & **Receive** \\ \hline SplitFed & 18.33 & 197.59 & 4.04 & 58.70 \\ (half data) & 9.36 & 98.86 & 3.97 & 29.40 \\ \hline PSL & 17.86 & 197.59 & 3.93 & 58.65 \\ (half data) & 8.79 & 98.89 & 3.88 & 29.42 \\ \hline AsyncSL & 4.58 & 28.24 & 1.84 & 8.39 \\ (half data) & 2.43 & 15.43 & 1.03 & 4.32 \\ \hline \hline \end{tabular}
\end{table}
Table 9: Communication cost per epoch on the MINST dataset when the number of clients is 3 [108].  
contains real-world objects, which require a sufficient amount of training data. Both SL paradigms show promising results, indicating that they can improve each client's model performance without exchanging the original data.
* The performance of local training and the two SL paradigms on non-IID data with varying \(\alpha\) is shown in Figure 11. On the MNIST dataset, the performance of Vanilla and local training decreases as \(\alpha\) decreases, but the overall performance of Vanilla is still higher than that of local training. When \(\alpha=0\), the accuracy of Vanilla also drops to a level similar to local training. On the CIFAR-10 dataset, Vanilla's performance is lower than expected; when \(\alpha\) is less than 0.5, it has a lower accuracy than local training. SplitFed performs well on both datasets, but on the CIFAR-10 dataset, there is a large drop in accuracy when \(\alpha\) is below 0.1. In the most extreme non-IID setting (\(\alpha=0\)), SplitFed achieves an accuracy of 64.73%. SplitFed also has difficulty in this case, but its performance is still better than the other two paradigms. SplitFed can mitigate this problem by increasing the frequency of aggregation.  
Generally, Vanilla SL helps clients improve performance when data are evenly distributed. However, for complex tasks and highly skewed non-IID distributions, SplitFed should be used to enhance the performance of local devices.