# FSCB-IDS: Feature Selection and Minority Class Balancing for Attacks Detection in VANETs
## 4 Results and Discussion
### Result Discussion
#### 4.2.2 Using TON-IOT Dataset  
TON-IOT dataset has also the problem of class imbalance which is shown in Figure 9. After using the SMOTE technique, this problem is solved as shown in Figure 10. The influence features present in our TON-IOT dataset are shown in Figure 11. We have selected the 15 best features to obtain better results. Figure 8: ROC curve for all applied techniques on the 15 best features (NSL-KDD). Figure 9: Attack class before applying the over sampling technique by using TON-IOT dataset. Figure 11: Influential features of detection attack on TON-IOT dataset. Figure 10: Attack class after applying the over sampling technique by using TON-IOT dataset. We first applied mutual information to select the 15 best features, as depicted in Figure 5. These features, including ts, src_ip, src_port, dst_ip, dst_port, service, duration, src_bytes, dst_bytes, conn_state, src_pkts, src_ip_bytes, and dst_pkts, were used in subsequent analyses. Furthermore, Figures 9 and 10 demonstrate the significant impact of employing the SMOTE technique to address class imbalance in the dataset. The improvement in class balance is evident, indicating the effectiveness of SMOTE in mitigating the issue. For evaluating the performance of our RF classifier, we compared it with LR, DT, KNN, and SVM techniques using the TON-IOT dataset. To provide a more comprehensive analysis, we examined the confusion matrices in Figure 12. Notably, our model exhibited outstanding performance with 0 FPR and FNR. Additionally, Figures 13 and 14 illustrates the ROC curve for all the applied techniques on the dataset. Our model achieved excellent results, attaining 100% AUC, which signifies its robustness in distinguishing between normal and intrusive instances. To provide a comprehensive assessment, we present the results in Table 4, which showcases the performance metrics of accuracy, recall, precision, and F1 score. Our model outperformed all other techniques, attaining 100% for each of these metrics. The second-best technique, decision tree (DT), also achieved 100% in all metrics. Figure 12: Confusion matrix of prediction using TON-IOT dataset. Figure 10.0 ROC curve on the 15 best features using TON-IOT Dataset. Figure 15 and 16 display the partitions of the attack class before and after applying the SMOTE technique. As depicted in Figure 16, the problem of class imbalance is effectively resolved, and the partitions are more balanced. Figure 17 showcases the influential features on our dataset, specifically the CICIDS2017 dataset. These features play a significant role in the performance of our model. Furthermore, the confusion matrices presented in Figure 18 demonstrate the correct implementation of our model, as indicated by the absence of FPR and FNR. In terms of performance evaluation, Table 5 presents the results obtained by the RF and SVM techniques. The RF technique yields satisfactory outcomes, with an accuracy, recall, precision, and F1 score of 0.999. On the other hand, the SVM technique exhibits  
Figure 10.1 Attack class after applying the over sampling technique using CICIDS2017 dataset. Figure 10.2 Influential features of detection attack on CICIDS2017 dataset. lower performance, with an accuracy of 0.180, recall of 0.340, precision of 0.160, and F1 score of 0.340. The results obtained allow us to conclude that the proposed framework is relevant and achieves significant performance. For the TON-IOT dataset, the performance evaluation results indicate that our RF classifier achieves exceptional results. The model exhibits 0 FPR and FNR as shown in the confusion matrices (Figure 12). The ROC curve (Figure 14) demonstrates that our model achieves 100% AUC, indicating its strong discriminatory power. Moreover, the results presented in Table 4 indicate that our RF classifier achieves the highest intrusion detection performance, with 100% accuracy, recall, precision, and F1 score. Moving on to the NSL-KDD dataset, the performance comparison reveals that our RF classifier continues to excel.