# Machine Learning Ship Classifiers for Signals from Passive Sonars
## 5 Results
### Comparison of Machine Learning Models for Classification
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Lofar**} & \multicolumn{2}{c}{**PCA**} & \multicolumn{2}{c}{**NLPCA**} \\ \cline{2-7}  & \(\mathbf{Acc}\) & \(\mathbf{\mathrm{SE}}\) & \(\mathbf{Acc}\) & \(\mathbf{\mathrm{SE}}\) & \(\mathbf{Acc}\) & \(\mathbf{\mathrm{SE}}\) \\ \hline \hline With Decimation & & & & & & \\ \hline MLP & 0.123 & \(5.96\cdot 10^{-2}\) & 0.437 & \(4.51\cdot 10^{-2}\) & 0.553 & \(6.08\cdot 10^{-2}\) \\ KNN & **0.996** & \(2.67\cdot 10^{-3}\) & 0.578 & \(7.91\cdot 10^{-2}\) & 0.578 & \(2.76\cdot 10^{-3}\) \\ Ridge & **0.958** & \(1.18\cdot 10^{-3}\) & 0.578 & \(7.91\cdot 10^{-2}\) & 0.578 & \(7.91\cdot 10^{-2}\) \\ SVM (OvO) & 0.517 & \(8.38\cdot 10^{-2}\) & 0.515 & \(8.35\cdot 10^{-2}\) & 0.515 & \(8.35\cdot 10^{-2}\) \\ SVM (OvA) & 0.467 & \(9.02\cdot 10^{-2}\) & 0.469 & \(9.54\cdot 10^{-2}\) & 0.469 & \(9.54\cdot 10^{-2}\) \\ Logistic Regression (OvO) & 0.932 & \(4.21\cdot 10^{-2}\) & **0.972** & \(4.42\cdot 10^{-3}\) & **0.972** & \(4.42\cdot 10^{-3}\) \\ Logistic Regression (OvA) & 0.917 & \(4.42\cdot 10^{-2}\) & **0.929** & \(4.08\cdot 10^{-3}\) & 0.929 & \(4.08\cdot 10^{-3}\) \\ LDA & 0.859 & \(8.39\cdot 10^{-2}\) & 0.859 & \(8.39\cdot 10^{-2}\) & 0.859 & \(8.39\cdot 10^{-2}\) \\ XGBoost & **0.997** & \(1.38\cdot 10^{-2}\) & 0.886 & \(1.38\cdot 10^{-2}\) & **0.997** & \(1.38\cdot 10^{-2}\) \\ \hline Without Decimation & & & & & & \\ \hline MLP & 0.363 & \(5.64\cdot 10^{-2}\) & 0.437 & \(4.51\cdot 10^{-2}\) & 0.437 & \(4.51\cdot 10^{-2}\) \\ KNN & 0.578 & \(7.91\cdot 10^{-2}\) & **0.996** & \(2.67\cdot 10^{-3}\) & **0.996** & \(2.76\cdot 10^{-3}\) \\ Ridge & 0.530 & \(6.94\cdot 10^{-2}\) & 0.530 & \(6.94\cdot 10^{-2}\) & 0.530 & \(6.94\cdot 10^{-2}\) \\ SVM (OvO) & 0.612 & \(3.44\cdot 10^{-2}\) & 0.0753 & \(1.18\cdot 10^{-2}\) & 0.0753 & \(1.18\cdot 10^{-2}\) \\ SVM (OvA) & 0.0896 & \(2.14\cdot 10^{-2}\) & 0.562 & \(4.27\cdot 10^{-2}\) & 0.529 & \(2.97\cdot 10^{-2}\) \\ Logistic Regression (OvO) & 0.917 & \(4.42\cdot 10^{-2}\) & **0.995** & \(2.70\cdot 10^{-2}\) & **0.995** & \(2.69\cdot 10^{-3}\) \\ Logistic Regression (OvA) & **0.995** & \(2.63\cdot 10^{-3}\) & **0.995** & \(2.53\cdot 10^{-3}\) & **0.995** & \(2.53\cdot 10^{-3}\) \\ LDA & **0.995** & \(4.69\cdot 10^{-3}\) & **0.995** & \(4.69\cdot 10^{-3}\) & **0.995** & \(4.69\cdot 10^{-3}\) \\ XGBoost & 0.996 & \(4.61\cdot 10^{-3}\) & 0.994 & \(5.75\cdot 10^{-3}\) & **0.997** & \(3.66\cdot 10^{-3}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 12: Mean accuracy (\(\mathbf{Acc}\)) and standard error (SE) for each model using Lofar, PCA, and NLPCA datasets, both with and without decimation. help in selecting appropriate models and preprocessing methods for tasks where the model stability and high accuracy are crucial, particularly in scenarios where data variance is a concern. In the area of feature vectors, the models generally showed few differences when comparing the Lofar, PCA, and NLPCA signals. Therefore, it is feasible to introduce a smaller feature vector to obtain good results. Consequently, a faster classifier can be created in a naval mission where the likely ships in the area are already known without the need for new signal incorporation. The most notable cases of increased accuracy were observed in LDA and MLP. Both algorithms aim to determine decision boundaries that separate different classes in the feature space. MLP can learn complex features, including nonlinear decision boundaries, while LDA assumes linear decision boundaries. The reduction in the input characteristics resulted in a significant difference between the two. The logistic regression model demonstrated solid performances in both scenarios--using the original data and the decimated version. XGBoost demonstrated consistent performances that surpassed all the others, and was therefore considered the ideal classifier compared with the other tested classifiers. Its robustness and effectiveness render it a strong candidate for various classification tasks.