# Finsformer: A Novel Approach to Detecting Financial Attacks Using Transformer and Cluster-Attention
## 1 Introduction
deployed a cryptomining scenario to train machine learning models for detecting malicious attacks on digital currency mining, achieving the capability to detect cryptomining attacks even in encrypted states [15]. [PERSON] et al. proposed a deep-learning-based system for the development of decentralized financial (DeFi) attack detection, DeFiScanner, with experimental results showing a true positive rate of 91% [16]. [PERSON] et al. proposed a deep-learning-based model for credit card fraud detection--a seven-layer neural network architecture--achieving an area under the ROC curve score of 99.1% [17]. [PERSON] et al. proposed a black-box attack scenario for financial institutions' transaction records, using adversarial training samples to test the robustness of models, aiding financial institutions in better utilizing deep learning models for transaction records [18]. [PERSON] et al. proposed a deep learning algorithm, DNN, to predict network attack patterns, reaching a prediction accuracy of 90.36%, beneficial for banks and other financial institutions to take preemptive security measures [19]. The research encompasses applications of deep learning techniques in detecting and defending against various cyberattacks, particularly in the context of smart cities [20], the internet of things [21], financial institutions [22], and other digital systems. These studies reflect the significant role and challenges faced by deep learning in the field of cybersecurity. In smart cities, challenges such as handling vast amounts of data and ensuring real-time responsiveness of models remain a significant issue [23], along with the need to address overfitting problems. This includes balancing the models' generalization capabilities with their sensitivity to attack behaviors [24]. In the internet of things, the challenge lies in adapting to new types of malware and reducing false positives, as well as enhancing the accuracy and adaptability of detection algorithms to cope with evolving attack methods [25]. The detection of phishing attacks is also a hot research topic [26], yet dealing with large-scale and dynamically changing network environments remains a significant challenge [27]. Additionally, security issues in the financial domain are receiving considerable attention,with the difficulty residing in rapidly adapting to and predicting new attack patterns within financial environments [28]. These studies indicate that, despite the vast potential of deep learning in the realm of cybersecurity, it also faces numerous challenges, including data processing, model generalization, and adapting to new types of attacks [29]. Finsformer proposed in this paper is a novel financial-system-attack detection model based on the Transformer and cluster-attention mechanism. The Transformer model, having achieved significant success in fields such as natural language processing (NLP), is highly suited for processing financial data sequences due to its powerful sequence-modeling capabilities. Meanwhile, the introduction of the cluster-attention mechanism, a novel aspect of this paper, aims to enhance the model's ability to identify attack patterns by effectively clustering data. The Transformer model was chosen for this paper because of its several advantages suitable for processing financial data. Firstly, the self-attention mechanism of the Transformer can capture long-distance dependencies in data, which is particularly important when dealing with financial time-series data. Secondly, compared to traditional deep learning models like convolutional neural networks (CNNs) and recurrent neural networks (RNNs), the Transformer offers higher efficiency and flexibility in processing sequence data. Additionally, the parallel processing capability of the Transformer makes it more efficient in handling large-scale data. The introduction of the cluster-attention mechanism is based on the particularity of financial data, which often contains diverse transaction patterns and user behaviors that traditional attention mechanisms may not fully capture.