# Research on Fault Diagnosis of Rotating Parts Based on Transformer Deep Learning Model
## 3 Results and Discussion
### Experimental Results and Analysis of XJTU-SY Dataset  
This experiment utilizes vibration data from the bearing acceleration life test at the 37 Hz 11 kN working condition in the Xi'an Jiaotong University (XJTU-SY) bearing fault dataset as experimental samples. The dataset includes three fault types: inner race, outer race, and cage, with corresponding labels set as 0, 1, and 2, respectively. Since the dataset is the whole life cycle data from health to fault state, the experiment only uses the data after the fault symptom as the training sample. The kurtosis of the data is greater than or equal to 3 for the first time as the basis for the beginning of the fault [28]. The training set of this dataset consists of 4874 samples, and the test set comprises 1219 samples. Analysis of Experimental Results  
For the XJTU-SY dataset, the experimental results of different models (the maximum values of the four indicators) are shown in Table 2 below. It can be seen that the proposed model is higher than other methods in the four evaluation indicators of fault diagnosis. The accuracy of the proposed model is 0.95078, which is higher than the accuracies of WDCNN, DRCNN, DilatedNN, and Vision Transformer by 0.09352, 0.02711, 0.01969, and 0.07711, respectively. In terms of the maximum accuracy metric, WDCNN has the worst performance, followed by Vision Transformer. This indicates that the unmodified Transformer framework is not effective for fault diagnosis. However, after adding the multi-head dilated convolution, the proposed model not only surpasses the performance of the Transformer itself but also outperforms several classic models commonly used for fault diagnosis. This suggests that the added modules and Transformer play a complementary role, enhancing the overall performance of the model. As shown in Figure 6, the test accuracy curves for the five models over 100 epochs are plotted. It can be observed that the proposed MultiDilatedFormer model initially lags behind the DRCNN and DilatedNN models but gradually surpasses them in the later epochs. From the change in curves, it can be observed that the prediction performance of the Vision Transformer model is the most unstable. Among them, WDCNN, DilatedNN, and the proposed MultiDilatedFormer model are relatively stable. From the enlarged view \begin{table}
\begin{tabular}{c c c} \hline \hline
**Module/Layer** & **Variable** & **Value** \\ \hline \multirow{2}{*}{Input Embedding Module} & Window Size & 105 \\  & Number of Patches & 21 \\  & \(D_{embed}\) & 512 \\ \hline \multirow{2}{*}{Multi-Head Self-Attention Layer} & Head & 8 \\  & Dropout & 0.2 \\ \hline \multirow{4}{*}{Multi-Head Dilated Convolution Layer} & Head & 8 \\  & Dropout & 0.2 \\ \cline{1-1}  & Kernel size & 3 \\ \cline{1-1}  & Dilation rates & (1,2,3) \\ \hline \multirow{2}{*}{Feedforward Network Layer} & Output size of FC1 and FC2 & 4 \(\times\)\(D_{embed}\) and \(D_{embed}\) \\ \hline \multirow{2}{*}{Feature Extraction block} & Number (Num) & 4 \\ \hline \multirow{2}{*}{Classifier Layer} & Output size of FC1 and FC2 & \(D_{embed}\)/2 and \(N_{class}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: MultiDilatedFormer model hyperparameter settings. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1 Score** \\ \hline WDCNN & 0.85726 & 0.85563 & 0.85752 & 0.85560 \\ DRCNN & 0.92367 & 0.92257 & 0.92227 & 0.92228 \\ DilatedNN & 0.93109 & 0.93010 & 0.93060 & 0.93020 \\ Vision Transformer & 0.87367 & 0.87465 & 0.87155 & 0.87212 \\ MultiDilatedFormer & 0.95078 & 0.95067 & 0.95065 & 0.95066 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Indicators comparison of XJTU-SY dataset. of the graph, it can be seen that the proposed model achieves the highest accuracy, and the training performance is also relatively stable. The normalized confusion matrix is introduced for visualization.