# In the phase of testing
## 4 Implementation and Results
### Results  
We applied our algorithm to the data set of the first five years (2009-2013), then we extended our data to confirm the stability of the results by adding data from the other five years (2018-2022). Figure 5 shows the performance of the different models (KNN, SVM, RF, DF, Log, and DL) on a dataset of five years (2009-2013). The RF and DT are superior in most of the parameters. DL also achieved a good result, but not better than RF; DL took more than 42 s for training while RF took less than 1 s.  
To confirm that, Figure 6 depicts the performance of the same models but in 10 years (2009-2013 and 2018-2022). The results here showed enhancement of DL results; however, it still not better than RF, which preserved the best performance and confirmed its stability. Of note, the DL took more than 120 s for the 8 years of data, while RF took less than 1 s.  
From Figure 5, the RF achieved the highest accuracy, \"99%\", and a maximum precision of \"1\". This means the margin of error is zero, and the probability of classified threat as  
Figure 5: Comparison among the selected models of all criteria (Data 2009–2013).  
Figure 6: Comparison among the selected models of all criteria (data 2009–2013 and 2018–2022).  
not threat is zero also. That is very important in sensitive or critical systems. For the recall value, RF achieved 99%, which means there is a very small margin for error classification for the non-threat case as a threat. There is no effect in this case in the proposed framework. Because the framework repeats the testing process periodically to enhance the accuracy and to find the level of threat in real-time.  
The F-score is the common factor that expresses recall and precision together. The results confirmed the superiority of RF and DT with KNN over the other models. Finally, the AUC and ROC present the significant superiority of RF, which is close to the maximum area, as shown in Figure 7. Tables 3 and 4 present the actual metrics values of each model for both datasets (Dataset 1 2009-2013, Dataset 2 2009-2013 and 2018-2022). The time of training per second is added to the tables.  
Finally, Figure 8 depicts the learning curve of each classification model. This figure proved the superiority of RF where the curve toward \(1\) and the gap between validation and training curve minimum. In addition, DL, KNN, and DT provided very good performance.  
As a justification for the superiority of RF, usually, DT and RF achieve higher performance than other models if the variables of the dataset are categorical or its values have a categorical tendency. That means that the values of the variables have certain thresholds that affect the result, in addition, the values of variables do not have complex relations. That returns to the nature of the weather in KSA. Thus, the DT and RF are suitable for this type of data. Moreover, the RF is used to achieve more stability if the dataset is changed, and  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**ML Models** & **TP** & **TN** & **FP** & **FN** & **Accuracy** & **Precision** & **Recall** & **F1-Score** & **Time** \\ \hline SVM & 565 & 697 & 6 & 125 & 0.906 & 0.989 & 0.819 & 0.896 & \textgreater{}1 s \\ KNN & 675 & 666 & 16 & 37 & 0.962 & 0.977 & 0.948 & 0.  
that is exactly what is proven by extending the dataset by an additional five years of data. In addition, the DL can achieve the best results if we provide more and more data. For that, the accuracy of DL is enhanced when we extend data from 5 years to 10 years. However, on another side, the DL required more time for updating the model (in the training phase) (see Tables 3 and 4).  
Figure 8: Learning curve of accuracy achievement of all ML models and the DL one.