## 2 Relevant Theoretical Models
### Transformer Feature Fusion
#### 2.3.2 Cross-Attention Mechanism  
To emphasize the time-domain characteristics of bearing rotation, it is essential to reduce the influence of external environmental noise. To this end, an attention mechanism is incorporated into the Transformer framework. This mechanism enables the model to focus on the operational signals of the bearing while effectively suppressing external interference. The specific calculation formula is shown in Equation (17):  
\[\left\{ \begin{aligned} F_{t}&=softmax(\frac{F_{t}F_{t _{Q}}^{T}}{\sqrt{d_{k}}})F_{t_{V}}\\ F_{f}&=softmax(\frac{F_{f}F_{t_{Q}}}{\sqrt{d_{k}}})F_{f_{ V}}\end{aligned} \right. \tag{17}\]where \(F_{f}\) and \(F_{t}\) represent the attention features of the time-domain signal and the frequency-domain signal, respectively, while \(F_{tK}\), \(F_{tQ}\) and \(F_{tV}\) are all features from the time-domain signal. \(F_{fK}\), \(F_{fQ}\) and \(F_{fV}\) are all characteristics from frequency-domain signals; \(d_{f}\) and \(d_{t}\) represent the characteristic dimensions of the signals in the frequency domain and the time domain, respectively; _softmax_ represents the activation function. During the simulation process, when the bearing operates under an ideal state, the time-domain signal will not be disturbed by external factors. However, when the bearing is actually operating, the time-domain signal will be disturbed by external factors, resulting in the time-domain signal containing a lot of other characteristic information. Therefore, to better extract modal information from the time series, a cross-attention mechanism is employed. During this process, the features of each mode are effectively fused to enhance representation quality. Through the cross-attention mechanism's processing, the model focuses on the relationships between different features to enhance the model's robustness. Through the cross-attention mechanism for time-domain and frequency-domain signals, the core of enhancing the anti-interference ability of the fault diagnosis model lies in deeply integrating the complementary advantages of dual-domain features and achieving dynamic adaptive filtering. This mechanism first acquires the time-domain waveform and frequency-domain spectral features through parallel feature extraction channels, and then uses the bidirectional cross-attention weight allocation strategy to construct a dual-domain feature correlation matrix. In this process, external interference signals lack consistent time-domain and frequency-domain correlation. The attention mechanism automatically suppresses their corresponding feature weights. The true fault features, due to their strong time-frequency coupling, will obtain a significant weight increase and form a dynamic perception filter based on feature correlation. Ultimately, this mechanism enhances the cross-domain consistency features and significantly improves the anti-interference ability against external factors. Figure 2 illustrates the feature fusion module's operation. The query matrix \(W^{Q}F_{f}\) of \(F_{f}\) is used to calculate the correlation with \(F_{t}\), thereby calculating the cross-concern of \(F_{f}\) against \(F_{t}\). This mechanism can enhance the information exchange between the two feature matrices, so as to facilitate better fusion of the information between the two feature matrices. The cross-attention mechanism and the self-attention mechanism may be used to extract the correlations between the points in the sequence. The correlations among the different points in different sequences are different, which can serve as the basis for our classification, as shown in Equation (18):  
\[ \begin{cases}Cross-Attention\Big{(}F_{f},F_{t}\Big{)}=softmax(\frac{Q_{f}(K_{t })^{T}}{\sqrt{d_{k}}})V_{t}\\ \hskip 28.452756 pt=softmax(\frac{(W^{Q}F_{f}(W^{K}F_{t})^{T}}{\sqrt{d_{k}}})W^ {V}F_{f}\end{cases}  \tag{18}\]  
where \(W^{Q}\), \(W^{K}\) and \(W^{V}\) are learning parameters; \(d_{k}\) is the query matrix \(Q\) dimension.