# Adaptive Semi-Supervised Algorithm for Intrusion Detection and Unknown Attack Identification
## 4 Experiments
### Model Evaluation  
Our algorithm model is constructed based on normal traffic data and cannot specifically identify individual attack types during actual testing. Therefore, the experiment primarily uses a binary classification approach of \"Normal\" versus \"Abnormal\". To further validate the model's effectiveness, we conducted independent binary classification experiments for each of the five attack types shown in the Figure 8 (DoS, PortScan, DDoS, BruteForce, WebAttack) against normal traffic (Benign). Additionally, we combined these five attack types into a single \"Anomalous\" category and conducted an overall binary classification experiment against normal traffic. This experimental design allows for a comprehensive evaluation of the model's performance in detecting normal traffic versus various types of anomalous traffic.  
Our TransGAN-IDS model demonstrated strong detection capabilities across different attack types, particularly excelling in FOR. It effectively identifies significant anomalies in traffic, such as DoS and DDoS attacks, showcasing its advantage in handling large-scale traffic anomalies. However, for attacks with relatively subtle features, such as BruteForce  
Figure 7: Confusion matrix.  
and WebAttack, while recall rates remain high, improvements in precision are still needed (Figure 9).  
Specifically, while the model achieved good accuracy and recall across most attack categories in the Kitsune dataset, attacks like ARP MitM exhibited relatively higher FOR values, indicating a more noticeable issue with false negatives for these types. This points to the need for further refinement, especially in reducing the FOR and improving the precision for these more challenging attack scenarios. Moving forward, enhancing detection capability in these areas will be a primary focus, as we aim to achieve a more balanced performance across all metrics.  
In Figure 10, we validate the effectiveness of our proposed algorithm by comparing it with several other intrusion detection algorithms related to the CIC-IDS2017 dataset, including DI-NIDS [26], M-CNN [27], Method [28], SGAN-IDS [29], and Method [30].  
In the experimental comparison, our proposed TransGAN-IDS algorithm demonstrated significant advantages across multiple metrics, particularly in recall rate. With a recall rate of up to 0.995, TransGAN-IDS clearly outperforms other methods such as DI-NIDS, M-CNN, and SGAN-IDS, highlighting its superior accuracy in detecting actual intrusion behaviors, especially under conditions of data imbalance. The high recall rate indicates that TransGAN-IDS effectively reduces false negatives, which is crucial for intrusion detection systems, ensuring that most attacks are promptly identified.  
Figure 8: Test results of the algorithm model for DoS, PortScan, DDoS, BruteForce, WebAttack, and overall testing results.  
Figure 9: The test results of the algorithm model for 9 attack types, including Fuzzing, Mirai, OS Scan, SSDP Flood, and ARP MitM, as well as the overall test results.  
Although some methods, such as SGAN-IDS, show slightly better performance in precision and F1-score, TransGAN-IDS maintains an impressive precision of 0.912 and an F1-score of 0.952, while still demonstrating exceptional recall capabilities. This balance underscores the robustness of our semi-supervised approach, which leverages both the Transformer and GAN architectures to handle complex and dynamic intrusion patterns, making it particularly well-suited for the evolving threat landscape in IoT environments.