## 3 Modeling and Results
### Development and Assessment of PSVE Model
Figure 5: Multilayer perceptron network. Here, \(\eta\) is learning rate, \(L\) is loss function, \(\frac{\partial L}{\partial\omega}\) is the gradient of the loss function with respect to the weight \(\omega\). Subsequently, we applied pruning to these four models by configuring appropriate hyperparameters for the grid search using a 10-fold cross-validation method. In cross-validation, the dataset is partitioned into multiple subsets, with one subset serving as the validation set in each iteration while the remainder constitutes the training set. The model's performance metrics on the validation set are computed for each iteration. This procedure is repeated multiple times, with each iteration employing a different subset as the validation set. Precision, Recall, F1 score and ROC_AUC were selected to evaluate the performance of the model. The equations for these indicators are provided below:  
\[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}}, \tag{5}\]  
\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}^{\prime}} \tag{6}\]  
\[\text{F1}=2\times\frac{\text{Precision}\times\text{Recall}}{\text{Precision}+ \text{Recall}^{\prime}}, \tag{7}\]  
ROC_AUC (Receiver Operating Characteristic Area Under Curve) is the area under the ROC curve. The ROC curve is drawn with the recall rate of the model as the ordinate and FP as the abscissa. The performance was considered poor with an ROC_AUC of \(<\)0.6, average with 0.6-0.7, good with 0.7-0.8, and very good with \(\geq\)0.8. TP (True Positive) represents the number of samples that the model correctly predicts to be positive. FP (False Positive) represents the number or proportion of negative-class samples that the model incorrectly predicts as positive-class samples. FN (False Negative) is the number of False Negatives, which represents the number of samples that the model incorrectly predicts to be negative but actually turns out to be positive. In the grid search, according to the characteristics of each model, we designed the parameter grid, which is mainly used to control the model complexity, prevent overfitting and underfitting, and improve the model's generalization ability. The final result is shown in Table 2, with the best parameters in red and the mean and standard deviation corresponding to the best parameter model. The baseline models before parameter tuning were put together to observe the performance improvement of the model. The final results are presented in Table 3, with the best in red and the second in blue; RF has the best performance in many indicators, but the gap between DT and it is actually very small, and KNN ranks second in Recall and F1 Score. It is worth mentioning that MLP failed to fit in the initial training; thus, it is not in the result without hyperparameter tuning (w/o HT). However, after parameter tuning, its performance has been greatly improved, as shown in \[Gini(T)=1-\sum_{i=1}^{c}p_{i}^{2}, \tag{2}\]0 Meanwhile, the performance improvement of other models is almost equal, so it is difficult to evaluate the advantages and disadvantages of each model from here. Therefore, in order to further understand the fitting of the model, a learning curve is drawn, which used AUC_ROC as an evaluation indicator. AUC_ROC comprehensively considers the tradeoff between TP and FP of the classifier under different thresholds, so it can evaluate the performance of the classifier more comprehensively. At the same time, it has good robustness to unbalance problems and is not easily affected by the unbalance of sample categories, so it can evaluate the performance of the model more comprehensively and objectively. The learning curve is shown in \[Gini(T)=1-\sum_{i=1}^{c}p_{i}^{2}, \tag{2}\]1