# LLM-Based Cyberattack Detection Using Network Flow Statistics
## 1 Introduction
Decoder-only models, such as GPT-3, excel at generating text, while encoder-only models, such as BERT, are better suited for tasks like text classification. Encoder-decoder models, such as T5 [26], combine the strengths of both approaches and can perform a wide range of tasks. However, we found only two publications on cyberattack detection models based on fine-tuning LLMs. [PERSON] et al. [27] used an encoder-only model, namely BERT, whereas [PERSON] et al. [28] compared the use of an encoder-only model and a decoder-only model, that is, BERT and GPT 2.0, respectively. [PERSON] et al. [29] presented an architecture based on BERT, called SecurityBERT, designed specifically for cyberthreat detection in IoT/IIoT devices. They only used a specific dataset for Internet of things (IoT), Edge-IIoTset. We present a new approach for the detection of cyberattacks, based on the fine-tuning of an encoder-decoder LLM pre-trained for NLP tasks, more concretely T5. This strategy allows us to evaluate the most suitable LLM-based architecture for the detection of cyberattacks. Beyond the architectural differences, another significant distinction between our approach and other LLM-based methodologies lies in the handling of data imbalance and the nature of the data used for training. We used undersampling and they used synthetic data. In contrast with [PERSON] et al. [29], our study focuses on a more general NIDS for broad cyberattack detection on standard network flow statistics, without the specific privacy-preserving layers tailored for IoT constraints. The experiments performed obtained promising results for the metrics selected (accuracy, precision, recall and F-score [30]) for the aforementioned task, using the smallest available size of the T5 model. The results are 99.84%, 99.94%, and 99.90%, respectively, for all metrics, applying our method to the CSE-CIC-IDS2018, CIC-IDS-2017, and BCCC-CIC-IDS-2017 datasets. These exceptionally high results, confirmed through rigorous stratified fivefold cross-validation and empirical analysis of training and validation loss curves, demonstrate robust generalization and the absence of overfitting. Considering all the results from the papers included in the bibliographic study, as shown in Section 4, our results demonstrate superior performance. The present paper describes a cyberattack detection system using a DL model. More specifically, we fine-tune an encoder-decoder model, for NLP, to obtain a new model focused on solving a completely different task, namely, the detection of cyberattacks. We believe that our strategy is a novel approach, as we found no studies on transformer-based encoder-decoder models. In any event, after comparing our results with other LLM-based and non-LLM Machine Learning (ML)-based strategies, we can assert that our system achieves superior performance. The structure of this paper is as follows: Section 2 details the most relevant datasets for cyberattack detection and introduces the design of our system, describing the training steps, the data preprocessing process, including data cleaning and data transformation. It also analyses the data cleaning results in depth, including undersampling and data splitting, and describes the fine-tuning process; Section 3 details the results of the experiments performed; Section 4 describes the papers identified regarding strategies based on LLM, and other strategies non-LLM based on ML. It also compares our results with those from the studies mentioned in the literature review, with a particular focus on each dataset. Finally, Section 5 presents the conclusions and proposals for future work.