# TSViT: A Time Series Vision Transformer for Fault Diagnosis of Rotating Machinery
## 4 Experimental Results and Evaluation
### Comparative Analysis  
We selected three deep learning models for comparing and validating the TSViT model for fault diagnosis: Deep Convolutional Neural Network with wide first-layer  
Figure 17: Feature visualization in different TSViT layers through t-SNE on the CWRU test set (**a**) raw signals (**b**) cnn (**c**) the first block (**d**) the second block (**e**) the third block (**f**) the fourth block (**g**) the fifth block (**h**) the sixth block (**i**) the seventh block (**j**) the eighth block (**k**) classification layer. Figure 18: Feature visualization in different TSViT layers through t-SNE on the XJTU test set (**a**) raw signals (**b**) cnn (**c**) the first block (**d**) the second block (**e**) the third block (**f**) the fourth block (**g**) the fifth block (**h**) the sixth block (**i**) the seventh block (**j**) the eighth block (**k**) classification layer. kernels (WDCNN) [34], Long Short-Term Memory (LSTM) [35], and CNN-LSTM [36]. These three models are widely applied in fault diagnosis. The details are presented in Table 7.  
The WDCNN consists of five convolution and pooling layers and two linear layers. The convolution kernel in its first layer has a wide width to extract features and suppress high-frequency noises. The WDCNN can achieve fault diagnosis based on raw vibration signals. LSTM is a specialized type of RNN designed to address the challenges of gradient vanishing and explosion encountered during training with long sequences [31]. The length of each signal sample is 2048, which is considered too long for LSTM. This length can pose challenges, as subsequent LSTM units may find it difficult to capture information from previous units. To address this issue, we reshaped the signal samples into 64 \(\times\) 32 matrices. The CNN-LSTM utilizes a CNN to extract local features from vibration signals and LSTM to learn the temporal dependencies among these features. The above three models are widely applied in fault diagnosis based on vibration signals. In our comparative analysis, we also evaluate the performance of our approach alongside the achievements of other works that have applied transformers to fault diagnosis. This includes Time Series Transformer (TST) [32], Efficient Convolutional Transformer (ECTN) [37], Integrated ViT [20], and MCF-1 DViT [19]. The ECTN and Integrated ViT transform vibration signals into time-frequency representation maps, while the MCF-1 DViT processes signal samples with a length of 1024. These methods employ different mechanisms compared to the TSViT model. Therefore, we directly use their experimental results from the literature for comparison. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Stage** & **WDCNN** & **LSTM** & **CNN-LSTM** \\ \hline \multirow{8}{*}{1} & Convolution & & Convolution \\  & (channels = 16, & & (channels = 32, \\  & kernel size = 64, & & (64 \(\times\) 32), & kernel size = 64, \\  & stride = 16), & Linear (32,192) & stride = 32), \\  & Batchnorm, & & & Flu, \\  & Relu, & & & Maxpool (4,2) \\  & Maxpool (2) & & & \\ \hline \multirow{8}{*}{2} & Convolution & & Convolution \\  & (channels = 32, & & (channels = 32, \\  & kernel size = 3, & & LSTM (192,192) & kernel size = 5, \\  & stride = 1), & Dropout (0.1) & \(\times\)8 & stride = 1), \\  & Batchnorm, & & & Flu, \\  & Relu, & & & Maxpool (4,2) \\  & Maxpool (2) & & & \\ \hline \multirow{8}{*}{3} & Convolution & & & Convolution \\  & (channels = 64, & & (channels = 64, \\  & kernel size = 3, & & kernel size = 3, \\  & stride = 1), & \(\times\)3 & Linear (128, num\_class) & stride = 1), \\  & Batchnorm, & & & Flu, \\  & Relu, & & & Avgpool (0) \\  & Maxpool (2) & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 7: The detailed structure of the comparison models. Table 8 presents the comparison of results with other methods, highlighting the superiority of the TSViT in fault diagnosis. The TSViT achieves the highest accuracy among these methods without any preprocessing for vibration signals. Notably, the transformer-based models outperform the other approaches overall.