# Exploring Multi-Channel GPS Receivers for Detecting Spoofing Attacks on UAVs Using Machine Learning
## 5 Evaluation and Results
### Feature Selection
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Feature Selection** & **Decision Tree** & **Random Forest** & **XGBoost** \\ \hline Before & 4.2450 s & 106.3031 s & 6.3194 s \\ After & 2.0497 s & 61.8106 s & 5.4666 s \\ \hline \hline \end{tabular}
\end{table} 
Table 14: Training time before and after applying feature selection in seconds. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Model** & **Accuracy \(\pm\) STD** & **Precision \(\pm\) STD** & **Recall \(\pm\) STD** & **F1-Score \(\pm\) STD** \\ \hline \multirow{3}{*}{Mission 1} & Decision Tree & 99.943 \(\pm\) 0.018 & 99.93 \(\pm\) 0.028 & 99.927 \(\pm\) 0.024 & 99.929 \(\pm\) 0.021 \\  & Random Forest & **99.979 \(\pm\) 0.007** & **99.977 \(\pm\) 0.010** & **99.973 \(\pm\) 0.011** & **99.975 \(\pm\) 0.008** \\  & XGBoost & 99.918 \(\pm\) 0.017 & 99.902 \(\pm\) 0.023 & 99.895 \(\pm\) 0.027 & 99.898 \(\pm\) 0.021 \\ \hline \multirow{3}{*}{Mission 2} & Decision Tree & 99.965 \(\pm\) 0.013 & 99.933 \(\pm\) 0.035 & 99.953 \(\pm\) 0.027 & 99.943 \(\pm\) 0.026 \\  & Random Forest & **99.984 \(\pm\) 0.009** & **99.951 \(\pm\) 0.035** & **99.979 \(\pm\) 0.020** & **99.965 \(\pm\) 0.019** \\  & XGBoost & 99.96 \(\pm\) 0.016 & 99.919 \(\pm\) 0.038 & 99.945 \(\pm\) 0.037 & 99.932 \(\pm\) 0.030 \\ \hline \multirow{3}{*}{Mission 3} & Decision Tree & 99.990 \(\pm\) 0.003 & 99.978 \(\pm\) 0.011 & 99.969 \(\pm\) 0.012 & 99.974 \(\pm\) 0.009 \\  & Random Forest & **99.995 \(\pm\) 0.001** & **99.991 \(\pm\) 0.006** & **99.984 \(\pm\) 0.008** & **99.988 \(\pm\) 0.005** \\  & XGBoost & 99.894 \(\pm\) 0.019 & 99.769 \(\pm\) 0.058 & 99.764 \(\pm\) 0.046 & 99.766 \(\pm\) 0.040 \\ \hline \multirow{3}{*}{All missions before Feature Selection} & Decision Tree & 99.972 \(\pm\) 0.006 & 99.953 \(\pm\) 0.015 & 99.941 \(\pm\) 0.017 & 99.947 \(\pm\) 0.015 \\  & Random Forest & **99.99 \(\pm\) 0.002** & **99.985 \(\pm\) 0.005** & **99.979 \(\pm\) 0.006** & **99.982 \(\pm\) 0.004** \\ \cline{1-1}  & XGBoost & 99.894 \(\pm\) 0.020 & 99.814 \(\pm\) 0.040 & 99.828 \(\pm\) 0.037 & 99.821 \(\pm\) 0.033 \\ \hline \multirow{3}{*}{All missions after Feature Selection} & Decision Tree & 99.976 \(\pm\) 0.005 & 99.962 \(\pm\) 0.012 & 99.947 \(\pm\) 0.015 & 99.954 \(\pm\) 0.012 \\ \cline{1-1}  & Random Forest & **99.99 \(\pm\) 0.002** & **99.984 \(\pm\) 0.005** & **99.98 \(\pm\) 0.006** & **99.982 \(\pm\) 0.004** \\ \cline{1-1}  & XGBoost & 99.894 \(\pm\) 0.022 & 99.817 \(\pm\) 0.043 & 99.828 \(\pm\) 0.040 & 99.822 \(\pm\) 0.036 \\ \hline \hline \end{tabular}
\end{table} 
Table 15: Comparative performance of decision tree, random forest, and XGBoost models on different mission datasets. Furthermore, our approach, which explores a GPS characteristic dataset to perform multiclass classification of GPS signals using tree-based models, outperforms the state-of-the-art methods based on binary classification [36] and those leveraging satellite imagery datasets [30] for detecting GPS spoofing attacks on UAVs. As shown in Table 16, X refers to studies that do not consider multi-attack classification, while \(\surd\) performs multi-attack classification.