# Research on Fault Diagnosis of Rotating Parts Based on Transformer Deep Learning Model
## 3 Results and Discussion
### Experimental Results and Analysis of CWRU Dataset
#### 3.5.2 Model Interpretability Discussion and Analysis  
This study introduces the visualization of feature heatmaps from the output of each layer to aid in understanding the decision-making mechanism of the model, thereby enhancing the credibility and interpretability of the model. The following is the visualization of the overall decision-making process of the model when inputting one sample with label 0 from the CWRU dataset.  
Figure 12a shows the input original vibration data sample, and the size is consistent with the model hyperparameter setting. Figure 12b is the feature map obtained by copying and expanding the original data. Each row has the same color, that is, the same value. Figure 12c, d is the feature maps obtained after input coding and position coding, respectively. It is obviously observed that, after the embedding of position coding, the feature distribution changes significantly, and the weight distribution is more dispersed. Figure 12e is the output after the multi-head self-attention layer. The weight distribution is more concentrated than before. Blue indicates a lower weight value, and the model begins  
Figure 11: The t-SNE cluster diagram of CWRU dataset: (**a**) WDCNN; (**b**) DRCNN; (**c**) DialetedNN; (**d**) Vision Transformer; (**e**) MultiDilatedFormer.  
to focus on some parts. Figure 12f is the output after multi-head dilated convolution. The fine-grained features are larger than before, and the model pays more attention to local features with smaller regions. Figure 12g is the feature vector of \((1,512)\) obtained by global average pooling, that is, patch sequence fusion. Finally, the feature vector shown in the right figure of (h) is output by the classifier. The closer to the red in the graph, the greater the classification score obtained. It is obvious that the predicted result of the sample is label 0, and the diagnosis result is consistent with the actual situation, which proves the effectiveness of the above layers in model decision-making. The left image of (h) is the bar graph obtained by classification. The highest positive value is the prediction result, and the prediction result is obviously labeled as 0.  
Figure 13 is the characteristic heat map of the multi-head mechanism in the multi-head self-attention layer (MSL) and the multi-head dilated convolution layer (MSL). Each attention head focuses on feature information in different subspaces of the input vectors. It can be observed that the color distribution for each attention head is distinct, with some heads having a predominance of blue, indicating low attention in that region, while others  
Figure 12: Model decision-making process visualization: (**a**) input sample, size: (1, 105); (**b**) after expand, size: (105, 105); (**c**) input embedding, size: (25, 512); (**d**) positional encoding, size:(25, 512); (**e**) multi-attention, size: (25, 512); (**f**) Multi–DilatedConv, size: (25, 512); (**g**) patch fusion, size: (1, 512); (**h**) output, size: (1, 10). The left is a bar chart, and the right is a heat map.  
have an increase in red, signifying high attention in that region. By carefully comparing (a), (b), (c), and (d), it can be observed that the Query, Key, and Value features in multi-head self-attention are linearly distributed in the vertical direction, while the D1 fine-grained higher weight distribution in the multi-head dilated convolution is more dispersed, indicating that multi-head dilated convolution pays more attention to small local features, while multi-head self-attention pays more attention to global features with larger regions.  
Figure 13: Visualization of multi-head mechanism: (a) query of MSL; (b) key of MSL; (c) value of MSL; (d) D1 after MDL.