# LLM-Based Cyberattack Detection Using Network Flow Statistics
## 4 Related Work
### Strategies Based on LLM  
In this paper, we present a new approach, namely, the detection of cyberattacks based on the fine-tuning of an encoder-decoder model pre-trained for NLP tasks. Regarding our approach, we found a limited number of papers related to NIDS based on an LLM, using the CSE-CIC-IDS2018 dataset, presented in this section. Two papers are based on an encoder-only model and the other on encoder-only and decoder-only models. Moreover, we include two papers based on the same line of research, but using a different dataset. [PERSON] et al. [28] introduced FlowTransformer, which is a transformer framework for flow-based NIDS. They used three datasets for the study of results: CSE-CIC-IDS2018, NSL-KDD and UNSW_NB15. They used different approaches and two well-known transformer architectures, GPT 2.0 and BERT. These are decoder-only and encoder-only LLMs, respectively. In contrast, our proposed approach uses the T5 model, which is based on an encoder-decoder architecture. For the CSE-CIC-IDS2018 dataset, using the GPT-based model, they obtained an accuracy of 95.86% and an F-score of 96.93%, and for the BERT-based model, they obtained an accuracy of 95.48% and an F-score of 95.80%. [PERSON] et al. [27] presented the Conditional Generative Adversarial Network (CGAN), used together with BERT, for intrusion detection. Beyond the architectural differences previously discussed, another significant distinction between our approach and other LLM-based methodologies lies in the handling of data imbalance and the nature of the data used for training. The CGAN was used to balance the dataset with data augmentation. We used original data, since we performed an undersampling of the majority class, while they used synthetic data. They presented results after testing their approach with the datasets NF-ToN-IoT-V2, NF-UNSW-NB15, and CSE-CIC-IDS2018. However, in their study, we did not find all the original types of attacks for the latter dataset, which is that used for our research. Finally, they prepared a comparison between the three aforementioned datasets using different methods. For the CSE-CIC-IDS2018 dataset, using their proposed approach, they obtained 98.22% accuracy, 98.25% precision, 98.22% recall, and 98.23% F-score. For the other methods, their accuracies are between 86.07% and 96.83%, precision rates between 85.91% and 97.46%, recall rates between 86.07% and 96.83%, and an F-score between 84.39% and 97.01%. The references mentioned in the previous two paragraphs are the only ones working on an LLM-based approach, for the detection of cyberattacks, using the same dataset as in the present study. After comparing both studies and our proposed approach, we can assert that our results are superior, as we explain in Section 4. [PERSON] et al. [29] presented an architecture based on BERT, called SecurityBERT, obtaining 98.2% accuracy and 98% precision, recall, and F-score. They reported an inference time of 0.15 s on an average CPU and a compact model size of 16.7 MB. They introduced a privacy-preserving BERT-based lightweight model designed specifically for cyber threat detection in IoT/IIoT devices. Their approach, while also leveraging a transformer architecture (BERT, an encoder-only model), includes a unique information encoding layer aimed at preserving data privacy, which is a critical concern in resource-constrained and sensitive IoT environments. This specialization for IoT/IIoT networks, driven by specific privacy requirements, distinguishes their work. In contrast, our study focuses on a more general NIDS, using an encoder-decoder model, T5, for broad cyberattack detection on standard network flow statistics, without the specific privacy-preserving layers tailored for IoT constraints. They only used the Edge-IIoTset dataset, which contains network traffic, but is focused on IoT. [PERSON] et al. [53] presented a model called BERTIDS, based on BERT. This paper was published in 2024 but only used the NSL-KDD dataset, published in 2009.