## 5 Experimental Results and Discussion
### Comparison
#### 5.2.3 The Current SOTA Models  
FAP-Prefix and the following models were trained on the EdNet-KT4 dataset, XuetangX, and orthografietrainer datasets. The current SOTA models in the LSDP task are introduced subsequently. Among these models, LSTM [19], GRU [20], and M3 PP [21] are temporal mining models. SVR [22] (intelligently optimized support vector regression) is a course dropout prediction model with an improved quantum particle swarm optimization (IQPSO) algorithm. In addition, MLP [23] and DAS [11] are models that have superior performance in recent studies on session dropout prediction. FAP-Prefix (prefix-length 3, ALBERT) is trained on the three datasets mentioned above. The comparative experimental results with the five SOTA models are presented in Figure 7.  
The experimental results show that FAP-Prefix achieves AUC of 0.9012, 0.8239, and 0.9193 as well as F1 scores of 0.8623, 0.8649, and 0.8648 on three different datasets, respectively. On the same datasets, FAP-Prefix significantly outperforms classical sequence mining models, LSTM, and GRU. The gating mechanism may not be effective enough when dealing with highly complex learning behavior features and implicit relationships. In addition, long sequence data may lead to a decline in performance because they require more time steps to process information. Finally, compared to FAP-Prefix, they lack the attention mechanism to identify the most important parts of the sequence. Obviously, SVR is far less effective in LSDP domains than in CDP tasks. The mapping of SVR is fixed. With limited data, SVR may fail to capture enough patterns for complex learning behavior features and relationships. Compared to M3 PP, FAP-Prefix achieves an increase in AUC by 0.1902, 0.1453, and 0.109, and in F1 scores by 0.1122, 0.1047, and 0.1026 on three different datasets, respectively. M3 PP models clickstream data as a continuous process, making it well-suited for time-dependent LSDP tasks. However, its state space grows exponentially when combining multiple sequences. In contrast, FAP-Prefix uses the transformer architecture to capture learning state features and shows better performance than  
Figure 7: Comparative experiments of different SOTA models and FAP-Prefix on different datasets. M3 PP when dealing with dense data. Among the SOTA models in the current LSDP task, MLP and DAS stand out as the latest and most outstanding models for predicting session dropout. Compared to MLP, FAP-Prefix achieves AUC improvements of 0.0976, 0.0697, and 0.051 and F1 score improvements of 0.0477, 0.0591, and 0.0592, respectively. MLP uses a multi-layer perceptron approach with a dual-matrix structure. It sequentially inputs all behavior features in the same session and updates parameters in matrix form during training. In contrast, prefix-tuning can learn the relationships between continuous behavior features. The latest sequence of continuous behavior features captures extensive forward-looking information, effectively reflecting the learner's current state for accurate dropout prediction. In comparison to DAS, FAP-Prefix improves AUC by 0.0909, 0.0794, and 0.0573 and F1 score by 0.0708, 0.0972, and 0.0917, respectively. The DAS model uses a single-layer transformer structure and multi-head attention, allowing the model to focus on information from different positions and vector representations. In comparison, FAP-Prefix adopts a 12-layer transformer structure with bidirectional training and prefix-tuning. FAP-Prefix enables better utilization of the limited data in LSDP tasks compared to DAS. FAP-Prefix demonstrates superior efficacy in the LSDP task, exhibiting higher predictive accuracy compared to current SOTA models. Table 4 shows the experiments between FAP-Prefix and SOTA models on the limited datasets. The results indicate that the advantages of FAP-Prefix are more pronounced in these datasets. Especially in the 10% and 1% datasets, FAP-Prefix demonstrates excellent performance improvements. This is because FAP-Prefix has higher data efficiency.