# FPE-Transformer: A Feature Positional Encoding-Based Transformer Model for Attack Detection
## 3 Materials and Methods
### Data Preprocessing  
A balanced dataset was created to address the problem of class imbalance between benign and attack samples of NSL-KDD. For binary classification, the dataset was split into two subsets based on their class labels: attack samples (1) and benign samples (0). The Borderline-SMOTE technique was applied to the attack class to mitigate the imbalance, generating synthetic samples by focusing on borderline instances, which are critical for classification tasks [62]. After oversampling the attack class, the k-Nearest Neighbors (k-NN) approach was used to downsample the benign class selectively. Specifically, the k-NN algorithm identified representative benign samples based on their proximity to the oversampled attack instances, ensuring a balanced yet representative subset of the benign class. The final balanced dataset was constructed by combining the oversampled attack samples with the downsampled benign samples. The dataset was then shuffled, and indices were reset to maintain randomness and consistency in further experiments. This preprocessing strategy resulted in a balanced dataset, effectively reducing the impact of class imbalance on model training while preserving the intrinsic characteristics of both classes. The subsampling of the benign class with the k-NN algorithm is as in Equation (1) [63].  
\[S_{B}=\{x_{i}\in C_{B}\,|\,\exists x_{j}\in C_{A},\,d(x_{i},x_{j})\leq d(x_{i},x_{k}),\,\forall x_{k}\in C_{B}\} \tag{1}\]  
\(S_{B}\) represents the subset of benign class examples, \(C_{B}\) represents all examples in benign class, \(C_{A}\) represents the set of examples in attack class, \(d(x_{i},x_{j})\) represents the Euclidean distance between data points \(x_{i}\) and \(x_{j}\), and \(k=3\) represents the number of nearest neighbors to be selected from the benign class. This formula calculates the distance of each data point \(x_{i}\) in the benign class to the points \(x_{j}\) in the attack class. It then performs the downsampling process by selecting the benign instances close to the \(k=3\) nearest attack class instances.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Category** & **Attack Types** & \begin{tabular}{c} **Train Sample** \\ **Count** \\ \end{tabular} & **Test Sample Count** \\ \hline Probe & \begin{tabular}{c} satan, ipsweep, \\ portsweep, nmap, \\ mscan, \\ snmpgetattack, \\ saint, snmpguess \\ \end{tabular} & 13,764 & 6639 \\ \hline R2L & \
\begin{tabular}{c} guess\_passwd, \\ buffer\_overflow, \\ warezmaster, imap, \\ rootkit, \\ loadmodule, \\ ftp\_write, phf, perl \\ \end{tabular} & 36,215 & 2227 \\ U2R &
\begin{tabular}{c} multihop, spy, \\ sqlattack, apache2, \\ processtable, \\ httpunnel, worm, \\ named, ps, \\ sendmail, xterm, \\ xlock, xsnoop, \\ udpstorm \\ \end{tabular} & 21,256 & 1551 \\ \hline \hline \end{tabular}
\end{table}
Table 5: _Cont._The attack types in the train and test sections of the NSL-KDD dataset are different. This can complicate evaluating the overall performance of models for multi-class classification and limit the model's ability to recognize different attack types. Therefore, the attack types are grouped appropriately in multi-class classification scenarios for more consistent comparison and analysis. In the NSL-KDD dataset, train and test data are grouped as \"normal\", \"Dos\", \"R2L\", \"Probing\", and \"U2R\" for multi-class classification. Table 5 presents the details of these groups for train and test data. Additionally, only numeric features of NSL-KDD were selected in the training and testing phase for binary and multi-class classification. Feature selection for the CIC-IDS2017 dataset was performed according to the importance weights given by the Random Forest (RF) algorithm. The \"Fwd Packet Length Max\",\"Fwd Packet Length Std\", \"Fwd Packet Length Mean\", and \"Bwd Packet Length Std\" properties were used for binary and multi-class classification.