## 4 CPS Framework Implementation and Experimentation
### Experiment and Result Analysis  
This part focuses on the performance of different DL methods rather than the setting of the hyperparameters. In the experiment, different methods in the same datasets used the same hyperparameters. The learning rate was 0.001 for spindle datasets. The learning rate was 0.001 from 0 to 29 epochs, 0.0001 from 30 to 59 epochs, and 0.00001 in the last 40 epochs for the bearing and gear datasets. The mini-batch size was 64 and the max number of epochs was 100. In the process of model training, Adam was used as the optimizer. Momentum is an important parameter of Adam and can accelerate the training process; it was set to 0.9 following the setup seen in [65]. The experiments were conducted on a system with i7 CPU @3.80 GHz and NVIDIA GeForce RTX 2060 SUPER. Python was used as a programming environment with pytorch. In the experiment, the maximum accuracy across all the epochs was chosen as the testing accuracy. In this experiment, we evaluated the accuracy when using different signal processing procedures and DL algorithms. In order to reduce the impact of the randomness, five trials were carried out for each experiment. The average performance of these methods, including LeNet, CNN, MSCNN, ResNet10, ResNet18, long short-time memory network (LSTM), and bidirectional LSTM (BiLSTM) [40], are shown in Tables 7-9 and Figures 9-11, in which the horizontal axis represents the combination of different DL algorithms and signal processing, the vertical axis represents the average accuracy, the histogram and black numbers represent the average accuracy value and the red box chart shows the dispersion at five times. STFT and ResNet obtained the best accuracy values of 92.08%, 95.12%, and 94.56%, respectively, in all three datasets. Figure 11: The results of the gear datasets. Figure 10: The results of the bearing datasets. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Signal** & \multicolumn{4}{c}{**Average Testing Accuracy \(\pm\) Standard Deviation (\%)**} \\
**Processing** & **MSCNN** & **CNN** & **LeNet** & **ResNet10** & **ResNet18** & **LSTM** & **BiLSTM** \\ \hline DMT & 94.06 \(\pm\) 0.46 & 88.45 \(\pm\) 0.19 & 64.80 \(\pm\) 4.42 & 93.66 \(\pm\) 0.37 & 93.08 \(\pm\) 0.51 & 77.95 \(\pm\) 1.20 & 77.24 \(\pm\) 0.25 \\ STFT & 94.41 \(\pm\) 0.23 & 91.77 \(\pm\) 0.82 & 88.24 \(\pm\) 0.35 & 94.56 \(\pm\) 0.27 & 93.98 \(\pm\) 0.15 & 92.24 \(\pm\) 0.37 & 92.55 \(\pm\) 0.42 \\ WPD & 83.97 \(\pm\) 1.88 & 86.27 \(\pm\) 1.54 & 74.18 \(\pm\) 2.18 & 90.50 \(\pm\) 0.61 & 89.53 \(\pm\) 0.43 & 83.69 \(\pm\) 0.80 & 84.13 \(\pm\) 0.40 \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Experimental results of the gear datasets. Figure 9: The results of the spindle datasets. As can be seen from Tables 7-9 and Figures 9-11, STFT always obtains the best accuracy in three datasets, the overall average accuracy is STFT > WPD > DMT. STFT and WPD can be used to gain the time-frequency characteristics of the data, and their accuracy is significantly higher than that of DMT. This result proves the necessity of the proposed PEM layer. The reason why the accuracy of STFT is higher than that of WPD is that the matter of how to choose the optimal wavelet basis function and wavelet decomposition level is challenging. For example, in the experiment, it was found that the accuracy of the CNN in spindle datasets was 72.24% for five-level wavelet decomposition under a DB1 wavelet basis function and 88.21% for six-level wavelet decomposition under a DB25 wavelet basis function, which represents an improvement of 15.97%. At the same time, we can see that the ResNet can always obtain the best accuracy. The order for the accuracy of the proposed hybrid modeling in three datasets with STFT is ResNet > MSCNN > BiLSTM > CNN > LeNet. Compared with other methods, the accuracy of LeNet is the lowest. As the depth of LeNet is the shallowest and the number of convolution channels here is the lowest, the features contained in the signal cannot be extracted completely.