# A Lightweight AI-Based Approach for Drone Jamming Detection
## 5 Model Performance
### Model Testing Phase
#### 5.2.2 Majority Rule Model Testing  
Having understood the nature of the problem, we decided to implement a Majority Rule (MR) system to reduce the number of false positives. The improved model would use the tuned Isolation Forest model's results and then pass them to a MR module, as shown in Figure 13.  
The logic behind the Majority Rule module is simple. The MR takes as input the classification results of the IF model and inserts them into a sliding window. If the data point is classified by the tuned IF as an anomaly, the MR module checks the contents of the sliding window. If the majority of the data points in the window are classified as anomalies, the data point is classified as an anomaly. If the majority of the data points in the window are classified as normal, then the data point is classified as normal (see Algorithm 2). ```
1: window \(\leftarrow[]\)
2:for each index in range(len(classification)) do
3:if length of window equals predefined window size then
4: window.pop(0)
5:endif
6: window.append(classification[index])
7:if current classification is OUTLIER and the count of OUTLIERS in the window
8: is less than or equal to half the window size then
9: classification[index] \(\leftarrow\) INLIERS
10:endif
11:endfor
12: return classification
```  
**Algorithm 2** Majority Rule Algorithm  
To determine the optimal value for the _window_size_ parameter, we test the model in the same way as in the hyperparameters tuning phase. The model parameters were set to the best performing values from the tuning phase (Table 8). Table 11 shows the tuning values for the _window_size_ parameter. From Figure 14, we can see that the _window_size_ parameter has a considerable impact on the model performance. \begin{table}
\begin{tabular}{c c} \hline \hline
**window_size** & **Values** \\ \hline Minimum & \(1\) \\ Maximum & \(100\) \\ Step & \(1\) \\ \hline \hline \end{tabular}
\end{table} 
Table 11: window_size tuning values. Figure 13: Majority Rule system diagram. The tuned Isolation Forest model results are passed to the MR module, which then provides the final classification. While accuracy and precision benefit from a larger window size, we can observe a linear decrease in recall as the window size increases. This is due to the nature of the Majority Rule, as a larger window size means that a larger number of anomalous data points must be present in the window in order to classify the current data point as an anomaly. The optimal value for the _window_size_ parameter was found to be **window_size = 39**, as it provides high accuracy and precision while maintaining a high recall. After defining the value for the _window_size_ parameter, we tested the model against the same input signal used for the standard tuned model (Figure 5). The results of the model classification are shown in Figure 15.  
In Tables 12 and 13, we can see a comparison between the tuned model and the MR model. Figure 14: Impact of window_size on several performance metrics (accuracy, precision, recall, F1 score). Most of the metrics initially benefit from a larger window size except for recall that suffers a linear decrease. Figure 15: Classification results from the MR Isolation Forest model. The results, as depicted in Figure 16, show that the Majority Rule model is effective in mitigating the high number of false positives that were present in the first version of the model. The MR integration resulted in an improvement in precision of 42.6%, in accuracy of 6.9% and in F1 score of 27.1%, while suffering only a less than 1% decrease in recall. In the original model, most of the false positives occurred due to the normal traffic samples containing a high number of isolated data points with similar RSS values to those of the jamming samples. The MR module addresses this by checking if the previous points were also classified as anomalies. This approach proves to be effective, as jamming attacks are rarely composed of isolated data points.