# TSViT: A Time Series Vision Transformer for Fault Diagnosis of Rotating Machinery
## 4 Experimental Results and Evaluation
### Hyperparameters
\begin{table}
\begin{tabular}{l c c c c c c c c c c c c} \hline \hline  & & \multicolumn{4}{c}{**Hyperparameters**} & \multicolumn{2}{c}{**FLOPS**} & \multicolumn{2}{c}{**Params**} & \multicolumn{2}{c}{**Average Accuracy**} \\ \cline{2-13}  & \(L_{p}\) & \(h\) & \(B\) & \(d_{MLP}\) & \(m\) & \(pe\) & \(dp\) & \(dp\) & (M) & \(\mathbf{\Omega}\) & \(\mathbf{\Omega}\) & \(\mathbf{\Omega}\) & \(\mathbf{\Omega}\) \\ \hline baseline & 32 & 8 & 8 & 768 & 192 & Yes & Yes & 309.88 & 2.39 & 100\% & 99.99\% & 100\% \\  & 64 & & & & & & 158.10 & 2.40 & 100\% & 99.96\% & 99.95\% \\ \(L_{p}\) & 128 & & & & & & 82.21 & 2.42 & 99.03\% & 99.98\% & 99.98\% \\  & 16 & & & & & & 613.44 & 2.38 & 100\% & 99.92\% & 99.99\% \\  & 4 & & & & & & 309.88 & 2.39 & 99.98\% & 99.93\% & 100\% \\ \(h\) & 6 & & & & & & 309.88 & 2.39 & 100\% & 99.93\% & 99.99\% \\  & 12 & & & & & & 309.88 & 2.39 & 100\% & 99.93\% & 100\% \\  & & 4 & & & & 155.73 & 1.20 & 99.98\% & 99.96\% & 100\% \\  & & 6 & & & & 232.81 & 1.79 & 100\% & 99.96\% & 100\% \\ \(B\) & & 10 & & & & 386.96 & 2.98 & 99.98\% & 99.96\% & 99.99\% \\  & 12 & & & & & 464.03 & 3.57 & 100\% & 99.96\% & 99.97\% \\  & & 256 & & & & 105.41 & 0.81 & 100\% & 99.94\% & 100\% \\ \(d_{MLP}\) & & & 512 & & & 207.65 & 1.60 & 100\% & 99.96\% & 99.98\% \\  & & & 1024 & & & 412.12 & 3.18 & 100\% & 99.93\% & 99.95\% \\  & & & 64 & & 103.29 & 0.80 & 99.85\% & 100\% & 100\% \\ \(m\) & & & 128 & & & 206.59 & 1.59 & 99.88\% & 99.98\% & 100\% \\  & & & 256 & & & 413.18 & 3.18 & 99.90\% & 99.94\% & 99.84\% \\ _pe5 dp_ & & & Yes & No & 309.88 & 2.39 & 99.80\% & 99.96\% & 100\% \\  & & & No & Yes & 309.88 & 2.39 & 99.98\% & 99.97\% & 99.97\% \\ \hline \hline \end{tabular} Note: \(pe\) is the abbreviation of position embedding and \(dp\) represents the dropout after \(pe\). FLOPS is the abbreviation of floating point operations per second.  
\end{table} 
Table 6: Influence of the hyperparameters on the performance of the TSViT. Within the transformer encoder layer, three crucial hyperparameters are the number of heads in MSA \(h\), the number of blocks \(B\), and the dimension of linear transformation in MLP \(d_{MLP}\). As indicated by Table 6, the computation load and parameter quantity remain constant, and the accuracy rises with an increase in \(h\). However, when \(h\) becomes  
Figure 15: Comparison of the influence of different hyperparameters on accuracy based on three datasets (**a**) patch size (**b**) dimension of time series patch embeddings (**c**) number of heads in MSA (**d**) number of blocks (**e**) dimension of linear transformation in MLP. excessively large, the feature subspace shrinks, hindering feature extraction. Figure 15c illustrates the results. Table 6 further indicates that the average accuracy increases with the enlargement of \(B\) or \(d_{MLP}\). However, this increase in \(B\) or \(d_{MLP}\) also results in a significant rise in the computational load and parameter quantity of the entire model. When these values become excessively large, overfitting may occur, leading to a decline in accuracy. Figure 15d,e illustrate the results.