# GNSS Spoofing Detection Based on Wavelets and Machine Learning
## 4 Results and Discussion
When the precision value is to close one, it means that the model has not missed any real positive results and is able to classify correct and incorrect sample labeling well. Low response (R < 0.5) means that the classifier has a large number of false negative samples that may be the result of an unbalanced class or unadjusted model hyperparameters. Ideally, R = 1, i.e., all positive samples are marked as positive by the classifier. It is ideal when all positively classified samples are really positive (P = 1) and, conversely, all positive samples are also classified as positive (R = 1). The performance parameters of the signal-type classification model for the different machine learning models for the db4 and db8 GPS wavelets and the different input classification data are shown in Table 2. The obtained classification values of several parameters for the SVM, KNN, RF-100, and RF-200 methods applied to the approximation coefficients based on the os2 and os4 datasets are presented. The performance parameters analyzed were the mean value of the model accuracy, F1-score, recall, and precision. Furthermore, a parameter that shows the time needed to run a particular model was added. From the table, we can see that, for example, the mean accuracy values were the same for both the  
Figure 3: ROC curves for SVM classification based on different types of input data for GPS datasets: (**a**) images, (**b**) images’ extracted features, (**c**) statistical and spectral features; and Galileo datasets: (**d**) images, (**e**) images’ extracted features, (**f**) statistical and spectral features. db4 and db8 wavelets. As for the classification with images as input data, the highest accuracy for both types of wavelets (db4 and db8) was obtained by the SVM model on the os2 dataset, with values of 98.94% and 99.43%, respectively. It can be concluded that the SVM with Gaussian (RBF) kernel achieved the best performance results because it implicitly handled local variations in signal power, critical for spoofing detection with small dB advantages such as os4 and os12. The results for both spectral and statistical image characteristics were all equally good, and the RF-100 and RF-200 models stood out for the statistical ones with an accuracy of 100% and F1, R, and P values that amounted to one. The only difference between the two models was in the runtime because RF-200 had more trees compared to RF-100, and it took longer to run. The number of trees was gradually increased and values above 200 trees did not give any significant accuracy improvements but only increased the execution time and computational complexity. Our model achieved stability with 100-200 trees, and adding more trees did not improve performance metrics. When OOB estimates and cross-validation results for the RFs were compared, cross-validation gave slightly better results than OOB in the case of weak spoofing attacks such as in the os4 and os12 datasets. For example, for the os4 dataset and an RF with 100 trees, the F1-score for OOB was around 0.7 while the F1-score for cross-validation had a value of 0.7742. For os2 and an RF with 100 trees, the F1-score for cross-validation was 0.9537 while for OOB, it was 0.91. The values of R and P were all greater than 0.7, which indicated good classification models. The runtime of models that had images as input was much higher compared to models that took numerical values as input. The maximum run time was 1202 s for the RF-200 model on the os2 dataset because that model was the most complex due to the number of trees. Figure 4: Precision–recall curves for SVM classification on the GPS os2, os3, and os4 and Galileo os10, os11, and os12 datasets based on different types of input data: (**a**) image (GPS), (**b**) images’ extracted features (GPS), (**c**) statistical and spectral features (GPS), (**d**) image (Galileo), (**e**) images’ extracted features (Galileo), (**f**) statistical and spectral features (Galileo).