# A Review and Experimental Evaluation on Split Learning
## 2 Taxonomy and Review of SL Paradigms
### Model Split-Only Approaches
In this paradigm, as shown in Figure 2b, the original model is divided into three parts: head \(\theta^{c_{h}}\), body \(\theta^{s}\), and tail \(\theta^{c_{1}}\), \(\theta^{c_{h}}\) and \(\theta^{c_{1}}\) are on the client side, while \(\theta^{s}\) is on the server side. The client only sends smashed data \(\mathbf{z}_{h}\) to the server; the server receives and forwards the smashed data for forward propagation and then sends the computed results back to the client. The client uses \(\theta^{c_{l}}\) to calculate the loss and gradient, updates the tail model with backpropagation, and sends the gradient to the server. The server updates \(\theta^{s}\) and finally sends the backpropagation result to the client to update \(\theta^{c_{h}}\). The above process ensures that only smashed data and gradients, rather than labels, are shared between clients and the server, thus avoiding information leakage caused by label sharing. Due to the exchange of smashed data and gradients between clients and the server in each batch, the communication cost of the U-shaped paradigm can be quite high. To address this issue, the Split Learning using Asynchronous Training (AsyncSL) paradigm [86] is proposed, which is a loss-based asynchronous training scheme that updates the client-side model less frequently and only sends/receives smashed data/gradients in selected epochs. The process involves a server state, as shown in Figure 3a. In State \(\mathcal{A}\), the clients send smashed data to the server and the server sends the gradient back. In State \(\mathcal{B}\), only the client sends smashed data to the server and the server does not send the gradient back. In State \(\mathcal{C}\), no information exchange occurs between the clients and the server. The state transition depends on the change in the loss function \(\Delta\mathcal{L}\), as shown in Figure 3a. Figure 2: Illustration of three model split-only SL paradigms: (**a**) PSL; (**b**) U-shaped; and (**c**) PC-SSL. Also, to reduce communication costs, the 3C-SL paradigm that combines circular convolution [100] with SL is proposed in [17]. As shown in Figure 3b, in this paradigm, clients use cyclic convolution to compress multiple features into a single one, which is decoded on the server side through cyclic correlation. Although this process may introduce noise, its impact on performance is mostly negligible. Sometimes, it can even result in regularization effects and improve performance. Similarly, deprune [89] is an adaptive compression-aware SL paradigm designed to improve communication efficiency. This paradigm utilizes pruning and transfer learning to train models with different compression levels for the data being transferred during SL. The motivation of the Peer-Coordinated Sequential Split Learning (PC-SSL) paradigm [87] is to enable each client to independently make accurate and generalized decisions without relying on communication with the server. Therefore, as shown in Figure 2c, the training process in this paradigm progresses sequentially from the first to the last user. And a cycle ends as the last user becomes the next user. Each step between a user's client and server reflects traditional local training; each step between the server of a user and the client of the next user represents a vanilla peer-to-peer SL. COMSPLIT [88] is a communication-aware SL paradigm designed for heterogeneous IoT environments. It enhances SL by adapting to various wireless channel conditions, integrating RNNs for time-series data, and supporting devices with different computational capacities. With an early exit strategy and improved channel modeling, COMSPLIT ensures robust and efficient SL that is demonstrated by water quality monitoring applications. The rapid development of Large Language Models (LLMs) marks a breakthrough in AI. A few studies have considered how to train LLMs in scenarios with limited computational resources using SL. [PERSON] et al. [90] proposed CARD, a method that combines LoRA-based fine-tuning with SL.