# FPE-Transformer: A Feature Positional Encoding-Based Transformer Model for Attack Detection
## 5 Conclusions  
This paper presents a novel Transformer-based attack detection framework. While existing Transformer-based models used for attack detection generally focus on hybrid structures and attention mechanisms, the FPE-Transformer model differs from these models with a feature-based feature positional encoding and ClassificationHead structure. This study used the known NSL-KDD and CIC-IDS2017 datasets to validate the FPE-Transformer model. Binary and multi-class classification were performed with these datasets, and the results of Transformer models with FPE, UPE, and PE were compared. Experimental results show that FPE-Transformer provides higher accuracy than Transformer models with other positional encoding methods in various attack types. The feature positional encoding in the FPE-Transformer model offers the advantages of feature-based customization, better generalization ability, and computational efficiency. Using separate positional encoding matrices for each feature, the model can independently learn each  
Figure 9: The comparison of CIC-IDS2017 multi-class classification test accuracy and loss values. (**a**) Loss value comparison for FPE, UPE, and PE in multi-class classification. (**b**) Accuracy value comparison FPE, UPE, and PE in multi-class classification. feature's order and timing. This is a great advantage, especially in cases where different features have different timing or ordering patterns, such as multi-feature time series. In scenarios where different features may exhibit different contextual relationships or timing patterns, applying positional encoding specific to each feature allows the model to learn better and generalize these differences, resulting in more accurate results, especially in multivariate datasets. In addition, ClassificationHead in FPE-Transformer allows the model to learn complex features by adding additional configurations such as linear layers and activation functions, thus producing effective and meaningful classification results. Future studies aim to adapt the FPE-Transformer model to robotic applications and develop it to support the integration of information from multiple data sources. FPE-Transformer can learn timing and contextual relationships specific to each data source with its feature-based positional encoding approach. Developing this model with multi-source data sources can provide higher accuracy and flexibility in attack detection and robotic systems' security. Additionally, the MHA mechanism can be optimized to make it more effective for attack detection. This mechanism can customize headers to model attack-specific behavior and, more precisely, analyze threat scenarios' different features. For example, the attention mechanism can be strengthened to distinguish anomalies over time to detect anomalous data patterns earlier and more accurately. In addition, the dynamic structures of current threats can be better understood with the information learned from past attack patterns, which can increase the adaptability and accuracy of the model. Conceptualization and methodology, H.C.Z. and E.N.Y. ; software, H.C.Z. ; validation, formal analysis, and investigation, H.C.Z. and E.N.Y. ; data curation and writing--original draft, H.C.Z. ; writing--review and editing, E.N.Y. ; visualization, H.C.Z. ; supervision, E.N.Y. All authors have read and agreed to the published version of the manuscript. This research received no external funding. Not applicable. Publicly available datasets were analyzed in this study.