## 4 Results and Analysis
### Prediction Results
#### 4.3.1 Prediction Results Based on Imbalanced Training Dataset  
Table 3 showcases the hyperparameter optimization outcomes for each classifier. The dataset includes 349 acoustic decoy samples and 899 submarine samples, totaling 1248 samples for the dataset. Encoding the acoustic decoy as 0 and the submarine as 1, Figure 13 shows the prediction result of the nine classifiers. In Figure 13, the red circle represents the true type of the samples, and the blue x represents the prediction results of different classifiers. It can be seen that the stacking ensemble classifier has the lowest number of misclassified samples and the best classification performance compared to other  
classifiers. It is worth noting that for SVM, KNN, and LR, the amount of submarine data misclassified is less than that of acoustic decoys. Taking LR as an example, only a few other submarine samples have been misclassified as acoustic decoy samples, while most of the acoustic decoy samples have been misclassified as submarines. This is because the dataset is imbalanced. Figure 13: Predictive results of each classifier based on imbalanced training dataset: (**a**) SVM; (**b**) KNN; (**c**) DT; (**d**) LR; (**e**) GBDT; (**f**) RF; (**g**) XGBoost; (**h**) stacking. \begin{table}
\begin{tabular}{c c} \hline \hline
**Classifiers** & **Hyperparameters** \\ \hline SVM & Kernel = \({}^{\prime}\)_rbf\({}^{\prime}\)_,_ \(C=0.8\)_,_ \(g=0.5\) \\ KNN & \(k=5\), weights = distance \\ DT & Min sample split = 4, min sample leaf = 3, criterion = \({}^{\prime}\)_gini_\({}^{\prime}\) \\ LR & \(C=0.9\), penalty = \({}^{\prime}\)_l_2’, solver = \({}^{\prime}\)_sag_\({}^{\prime}\) \\ GBDT & Learning rate = 0.1, n estimators = 100, max depth = 4, \\  & max features = 5, min sample split = 4, min sample leaf = 4 \\ RF & n estimators = 100, max depth = 6, max features = 5, \\  & min samples split = 6, min sample leaf = 6 \\ XGBoost & Learning rate = 0.1, n estimators = 100, max depth = 5, \\  & max features = 5, min sample split = 5, min sample leaf = 5 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Optimization results of hyperparameters based on the imbalanced training dataset. In order to further analyze the classification performance of each classifier, the indicator parameters mentioned in Section 4.1, which include accuracy, AUC, F1, PRE, REC, and kappa coefficients, are calculated to analyze each classifier. Figure 14 shows the 5-fold cross-validation accuracy, F1, and kappa results of each classifier, and Table 4 shows the _AUC_, _PRE_, and _REC_ results of each classifier. The following can be seen from Figure 14 and Table 4:  
* Compared with other classifiers, the stacking ensemble classifier has the best performance. For _acc_, the stacking ensemble classifier reaches 0.922 \(\pm\) 0.0402, which is higher than all other eight individual classifiers. For _AUC_, the stacking ensemble classifier reaches 0.958 \(\pm\) 0.0735, which is higher than all other eight individual classifiers. For _F1_, the stacking ensemble classifier reaches 0.948 \(\pm\) 0.0225, which is higher than all other eight individual classifiers. For _PRE_, the stacking ensemble classification algorithm reaches 0.941 \(\pm\) 0.0559, which is higher than all other eight individual classifiers. For _REC_, the stacking ensemble classifier reaches 0.957 \(\pm\) 0.0257, which is lower than the LR classifier (0.997 \(\pm\) 0.0072) and RF classifier (0.967 \(\pm\) 0.0479). It is worth noting that although the LR classifier has a higher REC, the PRE is not high. This phenomenon is due to the imbalanced dataset. * Tree-based classifiers, such as XGBoost, GBDT, and RF classifiers, have relatively good performance. These classifiers are also ensemble models based on decision trees. This indicates that ensemble classifiers have advantages in classification problems. We use these ensemble models as the base classifiers for the stacking ensemble classifier in this study. For these tree-based classifiers, XGBoost shows the best performance.