# LLM-Based Cyberattack Detection Using Network Flow Statistics
## 2 Materials and Methods
### Training the Cyberattack Detection System
#### 2.3.1 Data Preprocessing  
Data preprocessing is extremely important for the experiments to be carried out successfully. The main purposes of this task are: (a) to filter useless data by removing columns and rows that provide no extra information to the model or could cause worse results, and (b) to adapt data for it to be used directly in experiments. 1. Data cleaning The following paragraphs detail the data cleaning process:
2. Homogenization of column names: The selected dataset contains column names with all lowercase letters, words with the first letter capitalized, words with all letters capitalized, words separated by blanks or with the underscore character, and most names start with a blank space. We decided to remove initial blanks and to homogenize column names. To achieve this, all leading and trailing blanks were removed, all blanks were replaced by an underscore, and all letters were changed to upper case. 3. Removal of useless columns: The selected dataset contains a number of useless columns, since they are available for just a few rows of data, and are located in a single file. This involves missing values for most records. These columns are: (a) FLOW_ID identifies the flow, (b) SRC_IP identifies the source machine by its IP address, (c) SRC_PORT identifies the port in the source machine, and (d) DST_IP identifies the destination machine by its IP address. With the exception of the columns mentioned above, which were removed, the rest of the columns were maintained. 4. Removal of columns with unique values: Columns with unique values are not useful for model building as they provide no extra information. To carry out this process, the standard deviation of all columns is calculated, and if the deviation is 0, it means that all values are equal. If the deviation is close to 0, it means that almost all values are equal, and, in fact, it would also contribute nothing to the future model. In the experiments, we compared the number of columns with a standard deviation of 0 and with a very low standard deviation, for example, 0.01, and applying both filters, we obtained the same columns to be removed. Hence, we decided to eliminate all columns with a standard deviation equal to 0, since for both filters, we would have to remove the same columns. 5. Removal of columns with high correlation: Columns with high correlation were removed because they are not useful for the future model. The steps are as follows: (a) calculate the correlation matrix, (b) obtain the upper triangular matrix, (c) list the columns whose correlation is greater than 0.95, and (d) remove the columns with high correlation. 6. Removal of infinite, empty, and null values: Infinite, empty, and null values are useless, since they provide no additional information to the model. Therefore, rows containing such values were deleted. 7. Removal of duplicate rows: Finally, duplicated rows have to be removed, since they contribute nothing to our dataset, a common step in a data cleaning process. 8. Data transformation The features contained in the dataset, after cleaning, cannot be used directly with the T5 model, due to its architecture. Therefore, we apply two transformations:
9. Preparation of input strings: The T5 model, having been pre-trained on diverse NLP tasks, inherently requires string-based input for its \"source_text\" field. Our dataset, however, comprises 44 numerical features representing network flow statistics. To bridge this modality gap and enable the T5 model to process network flow data, we developed an abstract artificial language. This innovative design involves a two-step transformation process for each row of numerical data:  
* Feature-to-text conversion: Each numerical feature is individually converted into its string representation. * Sequential concatenation: These string representations of the features are then sequentially concatenated to form a single continuous input string.