# Finsformer: A Novel Approach to Detecting Financial Attacks Using Transformer and Cluster-Attention
## 4 Results and Discussion
### Ablation Study on Cluster-Attention Mechanism  
In the ablation study of cluster-attention mechanism within this article, the focus was on exploring the impact of different attention mechanisms on the performance of models for detecting financial attack behaviors. The experiments, by comparing the performance of models without attention mechanism, with multi-head attention mechanism, and with cluster-attention mechanism across key metrics such as precision, recall, and accuracy, provided an in-depth analysis of the strengths and limitations of each mechanism, thereby demonstrating the effectiveness of cluster-attention in specific applications, as shown in Table 2.  
The results indicated that models lacking an attention mechanism performed relatively poorly across all metrics. This was mainly attributed to their inability to dynamically focus on specific parts of the data, particularly in processing complex sequential data such as financial transactions. This static approach to information processing limited their capacity to capture key patterns and relationships in financial transaction data, resulting in less accurate identification of attack behaviors. The introduction of the multi-head attention mechanism significantly improved model performance. By simultaneously processing multiple parts of a sequence and calculating the interrelationships between different parts, the multi-head attention mechanism effectively enhanced the model's understanding of complex relationships. This indicated that the mechanism was capable of effectively handling complex patterns in financial transaction data, especially excelling in capturing associations between transactions, thereby improving the precision in detecting attack behaviors. Finally, the cluster-attention mechanism exhibited the best performance in the experiments. This outcome reflected that cluster-attention, by clustering data based on  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Attention Mechanism** & **Precision** & **Recall** & **Accuracy** & **F1-Score** \\ \hline No Attention [36] & 0.83 & 0.79 & 0.82 & 0.81 \\ Multi-head Attention [39] & 0.90 & 0.91 & 0.90 & 0.90 \\ Cluster-attention & 0.97 & 0.94 & 0.95 & 0.95 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Detection results of different attention mechanisms.  
Figure 5: Curves of detection results. Orange line is the training, blue one is the validation.  
feature similarity and then calculating attention weights on these clusters, focused attention more on key areas within the data. Through precise cluster analysis, the cluster-attention mechanism was able to more accurately focus on important patterns and relationships in the data, particularly enhancing the ability to identify anomalous behaviors in financial data.  
In summary, the experimental results clearly showed the roles and effects of different attention mechanisms in the task of detecting financial attack behaviors. Models without an attention mechanism showed poorer performance due to a lack of focus on important parts of the data. The multi-head attention mechanism significantly enhanced model performance by providing dynamic attention to the data. The cluster-attention mechanism, on the other hand, further optimized the allocation of attention through precise cluster analysis, significantly improving the accuracy and efficiency of the model in processing complex financial data. These findings not only prove the potential application of the cluster-attention mechanism in the domain of financial attack detection but also provide valuable references for future research in similar fields.