# A Review and Experimental Evaluation on Split Learning
## 2 Taxonomy and Review of SL Paradigms
### Model Split-Only Approaches
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Type** & **Paradigm** & \begin{tabular}{c} **Commun. Cost** \\ **Reduction** \\ \end{tabular} & \begin{tabular}{c} **Label** \\ **Protection** \\ \end{tabular} & \begin{tabular}{c} **Data** \\ **Heterogeneity** \\ \end{tabular} & \begin{tabular}{c} **Commun. or Comput.** \\ **Heterogeneity** \\ \end{tabular} \\ \hline \multirow{7}{*}{Model Split-only} & Vanilla [13] & \(\times\) & \(\times\) & \(\times\) & \(\times\) \\  & U-shape [13] & \(\times\) & \(\surd\) & \(\times\) & \(\times\) \\  & PSL [49] & \(\times\) & \(\times\) & \(\times\) & \(\times\) \\  & AsyncSL [86] & \(\surd\) & \(\times\) & \(\times\) & \(\times\) \\  & PC-SSL [87] & \(\times\) & \(\surd\) & \(\times\) & \(\times\) \\  & COMPSLIT [88] & \(\surd\) & \(\times\) & \(\times\) & \(\surd\) \\  & deprune [89] & \(\surd\) & \(\times\) & \(\times\) & \(\times\) \\  & CARD [90] & \(\surd\) & \(\times\) & \(\times\) & \(\surd\) \\ \hline \multirow{7}{*}{\
\begin{tabular}{c} Weight \\ Aggregation-based \\ \end{tabular} } & SplitFed [46] & \(\times\) & \(\times\) & \(\surd\) & \(\times\) \\  & FSL [91] & \(\surd\) & \(\surd\) & \(\times\) & \(\times\) \\  & ParallelSFL [92] & \(\surd\) & \(\times\) & \(\surd\) & \(\surd\) \\  & FeSTA [16] & \(\times\) & \(\surd\) & \(\times\) & \(\times\) \\  & FeSViBS [93] & \(\times\) & \(\surd\) & \(\times\) & \(\times\) \\  & 3C-SL [17] & \(\surd\) & \(\times\) & \(\times\) & \(\times\) \\  & HSFL [94] & \(\surd\) & \(\times\) & \(\times\) & \(\surd\) \\  & CPL [18] & \(\surd\) & \(\times\) & \(\times\) & \(\surd\) \\  & DFL [69] & \(\surd\) & \(\times\) & \(\times\) & \(\surd\) \\  & CHEESE [52] & \(\surd\) & \(\times\) & \(\times\) & \(\surd\) \\  & SplitFedZip [95] & \(\surd\) & \(\times\) & \(\times\) & \(\times\) \\  & SplitLLM [96] & \(\surd\) & \(\times\) & \(\times\) & \(\times\) \\  & FJORD [85] & \(\surd\) & \(\times\) & \(\surd\) & \(\surd\) \\  & SAFL [97] & \(\surd\) & \(\times\) & \(\surd\) & \(\times\) \\ \hline \multirow{7}{*}{
\begin{tabular}{c} Intermediate Data \\ Aggregation-based \\ \end{tabular} } & SGLR [58] & \(\times\) & \(\times\) & \(\surd\) & \(\times\) \\  & LocFedMix-SL [98] & \(\times\) & \(\times\) & \(\times\) & \(\times\) \\  & CutMixSL [59] & \(\surd\) & \(\times\) & \(\times\) & \(\times\) \\  & Mix2 SFL [60] & \(\surd\) & \(\times\) & \(\surd\) & \(\times\) \\  & TSL [99] & \(\surd\) & \(\times\) & \(\times\) & \(\times\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of existing SL paradigms along four dimensions. Here, “\(\surd\)” and “\(\times\)” indicate whether an SL paradigm takes into account the corresponding dimension in its design or not. Peer-to-Peer: The server sends the address of the previous client to the current client to transfer the model weights, as depicted in Figure 1b. Synchronous training in VSL can result in significant delays. To address this issue, the Parallel Split Learning (PSL) paradigm [49] has been proposed, as shown in Figure 2a. In PSL, the weights of the client model are not shared. The \(k\)-th client stores its own local model \(\theta_{k}^{c}\). Therefore, all clients can initiate the forward propagation process and transmit the smashed data \(\mathbf{z}_{k}\) to the server in parallel. Once the forward and backward propagation processes are completed, the server returns the gradients \(\mathbf{g}_{k}\) derived from \(\mathbf{z}_{k}\) to the \(k\)-th client, thereby reducing the training time. Therefore, the objective function in PSL can be denoted as  
\[\min_{\Theta^{c},\Theta^{c}}\sum_{k=1}^{K}\mathbb{E}_{D_{k}}\mathcal{L}(\theta ^{s}(\theta_{k}^{c}(X_{k})),y_{k}), \tag{1}\]  
where \(\Theta^{c}=\{\theta_{k}^{c}|k\in[K]\}\). In both VSL and PSL, clients send the labels of the samples to the server, which may cause label leakage. To address this problem, the U-shaped paradigm [13] has been introduced.