# FPE-Transformer: A Feature Positional Encoding-Based Transformer Model for Attack Detection
## 1 Introduction  
As the reliance on interconnected systems grows across various sectors, the need for robust cybersecurity mechanisms has become increasingly critical. Modern systems, integrated with complex networks and control architectures, are highly vulnerable to diverse cyberattacks. Such attacks can hinder operations, jeopardize data integrity, and result in substantial financial losses, reputational harm, and potential threats to public safety in critical settings. Effective attack detection systems are essential to prevent unauthorized access, mitigate data breaches, and safeguard the functionality of systems against manipulations that could compromise their operations. Addressing these cybersecurity challenges is vital for ensuring the reliability and resilience of today's interconnected infrastructures. Deep learning (DL) models have gained popularity by providing successful results in attack detection tasks [1]. DL models are more suitable for processing high-dimensional and complex features than classical machine learning (ML) models [2]. DL models such as Long Short-Term Memory (LSTM), Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), Bi-Directional-LSTM (Bi-LSTM), and Generative Adversarial Networks (GANs) are widely used in attack detection tasks. In [3], the LSTM model was used to detect replay and amplitude shift attacks, while in [4], it was used to detect falsedata injection attacks. Similarly, [PERSON] et al. applied an LSTM-RNN hybrid model for attack classification on the NSL-KDD dataset, achieving improved accuracy in binary and multi-class classification tasks [5]. [PERSON] et al. developed an approach for real-time classification using an innovative word embedding mechanism and LSTM model at the packet level to achieve faster attack detection results [6]. Another study utilized a CNN-LSTM hybrid model to detect BotNet attacks [7]. In addition, the CNN-Gated Recurrent Unit (GRU) hybrid model was proposed to improve the classification accuracy and class imbalance problem, and high accuracy rates were achieved on various datasets [8]. In another study on CNN-based attack detection, the performance of CNN and CNN-LSTM models for multi-class categorization of network attacks is discussed [9]. The effectiveness of these models is tested on the NSL-KDD dataset. The Bi-LSTM model, with its iterative learning capability, has been effectively used for tasks such as malware detection, VPN encapsulation recognition, and Trojan classification [10]. Additionally, the GAN model, which addresses the challenge of low attack detection accuracy due to limited labeled data, expands the training dataset by continuously generating samples, thereby significantly improving accuracy [11]. LSTMs and RNNs perform well in time series data by leveraging historical information; however, they may struggle to capture long-term dependencies and often encounter issues like gradient vanishing during training. While CNNs can extract spatial features effectively, they may be limited when dealing with time series or sequence-based data. Bi-LSTMs improve analysis by capturing both forward and backward dependencies, yet their high computational cost can impact performance, especially with long sequences. GANs are valuable for attack simulation and data generation, but their training process can be challenging to maintain, and they may lead to high false positive rates. In contrast, Transformer models excel at learning both short-term and long-term patterns by effectively capturing long-term dependencies through their self-attention mechanism [12]. Their parallel processing capabilities make them faster and more scalable for large datasets. By analyzing the entire sequence regardless of input size, Transformers avoid issues like gradient vanishing in sequence data, offering enhanced accuracy, scalability, and flexibility for detecting cyberattacks.