# Finsformer: A Novel Approach to Detecting Financial Attacks Using Transformer and Cluster-Attention
## 4 Results and Discussion  
### Attack Behavior Detection Results  
The primary objective of the experiments described in this article was to evaluate the performance of various deep learning models in the task of detecting financial attack behaviors. The experiments conducted a thorough analysis of the strengths and limitations of each model by comparing their performance across three key metrics: precision, recall, and accuracy. The results of the experiments are presented in Table 1 and Figure 5.  
It was observed that recurrent neural networks (RNNs) exhibited relatively weaker performance in this task. This is primarily attributed to the challenges RNNs face in handling long-term dependencies, limiting their ability to recognize complex patterns in financial transaction data. Long short-term memory (LSTM) networks, as an improvement of RNNs, effectively address the long-term dependency issue through the introduction of gating mechanisms, thereby showing better performance in recognizing and memorizing complex patterns in financial transaction data. The Transformer model outperformed both RNN and LSTM, benefiting from its self-attention mechanism that processes all elements of a sequence simultaneously, effectively capturing global dependencies. This ability enables the Transformer to more accurately capture complex relationships between various transactions in financial data, thus enhancing the accuracy of attack detection. The BERT model, a pre-trained language model based on the Transformer, excelled across all metrics due to its strong contextual understanding capabilities and the advantage of large-scale pre-training. This indicates BERT's effectiveness in handling complex contextual information, which is particularly important for identifying financial attack behaviors. Ultimately, the Finsformer model achieved the best performance on all evaluation metrics. This can be attributed mainly to its innovative cluster-attention mechanism, which, by performing cluster analysis on financial transaction data, captures key patterns in the data more accurately. Compared to traditional attention mechanisms, the cluster-attention  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **Precision** & **Recall** & **Accuracy** & **F1-Score** \\ \hline RNN [38] & 0.83 & 0.81 & 0.82 & 0.82 \\ LSTM [41] & 0.88 & 0.86 & 0.87 & 0.87 \\ Transformer [39] & 0.90 & 0.88 & 0.89 & 0.89 \\ BERT [46] & 0.93 & 0.91 & 0.92 & 0.92 \\ Finsformer & 0.97 & 0.94 & 0.95 & 0.95 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Detection results of different models.  
mechanism focuses more on significant areas in the data, significantly enhancing the accuracy in identifying anomalous behaviors.  
In summary, the experimental results not only demonstrate the characteristics and performance of different models in the task of detecting financial attack behaviors but also confirm the exceptional capability of Finsformer in handling such tasks from both mathematical and machine learning perspectives. These findings provide valuable insights and references for future research in the field of financial security.