# LLM-Based Cyberattack Detection Using Network Flow Statistics
## 4 Related Work
### Comparative Analysis
#### 4.3.1 Using the CSE-CIC-IDS2018 Dataset  
Sections 4.1 and 4.2 contain details on the publications included in this comparative study. Table 10 shows the results published in these papers and in ours, after applying our approach to the CSE-CIC-IDS2018 dataset. Beyond standard metrics such as accuracy, precision, recall, and F-score, this comparison critically includes the FPR and FNR, which are paramount for assessing the operational viability of an IDS. This table is divided into three parts by dark horizontal lines. The first part, with only one row, contains the results of our approach. Our approach, utilizing the T5 model, achieves a leading accuracy of 99.84%, demonstrating its strong overall performance. More importantly for real-world deployment, our model exhibits a highly competitive FPR of 0.27% and a remarkably low FNR of 0.02%. The second part shows the results of two approaches based on LLM and the same dataset we used. Our results are at least one percentage unit better than the others, for the metrics of accuracy, precision, recall, and F-score. Therefore, our approach performs better than those in the aforementioned papers, and it is worth noting that we selected the smallest size of the available snapshots of the T5 model. Regarding FPR and FNR, our system demonstrates a robust balance. For instance, while [PERSON] et al. [28], GPT 2.0 shows a lower FPR of 0.11%, and their overall accuracy (95.86%) is considerably lower, suggesting a trade-off in broader detection capability. Conversely, their BERT model yields a 0.47% higher FPR than ours. Finally, in the last part of the table mentioned above, we can find several references regarding IDS-related models that used ML techniques, non-LLM, and the same dataset we used. Our results are again better. Considering that we selected the smallest available size of the T5 model, we can assert that our results for all metrics are better than most of the papers found. Only two approaches, that is [20,21], obtained superior accuracies, and only another, namely [13], obtained slightly better precision. In the case of [PERSON] et al. [21], although accuracy is better, for the other metrics, their values are poorer than ours. In this sense, [PERSON] [56] affirms that accuracy can be a misleading metric for imbalanced datasets. Therefore, our system can be said to perform better. As for [PERSON] et al. [20], although their accuracy is higher, this alone is not informative without other metrics; hence we cannot conclude their approach is more effective than ours. Regarding FPR and FNR, [PERSON] et al. (2020) reported an FPR of 0.89%, and [PERSON] et al. (2019), with various ML models, reported higher FPR values, ranging from 0.42% to 0.53%. Our model also compares favorably with [PERSON] et al. (2019), where our FNR is significantly lower, indicating fewer missed attacks despite a slightly higher FPR. Notably, [PERSON] et al. (2019) achieves a lower FPR, but full comparative metrics for a balanced assessment are not available. Figure 5 provides a detailed visual analysis of the performance of our proposed approach and other comparative strategies, illustrating the difference of each metric relative to the absolute best value observed within each dataset CSE-CIC-IDS2018. For metrics not reported in the original comparative studies, a value of -1 is assigned for visualization purposes. This clearly distinguishes non-comparable data points, indicating that the original work did not provide these specific performance indicators, rather than implying a negative performance. A shorter bar or one closer to zero indicates that the proposal achieves the best performance or is very close to it for that specific metric. We can observe that for accuracy, precision, recall, and F-score metrics, our T5 approach consistently positions itself as one of the best or the outright best, exhibiting minimal (or zero, as for recall and F-score) differences compared to the maximum value achieved.