# Continuous Wavelet Transform and CNN for Fault Detection in a Helical Gearbox
## 5 Experiments and Discussions
### Experiments with the Proposed Defect Detection Models  
All four classification models (Mcwt_x, Mcwt_y, Mcwt_z, and Mcwt_xyz) were built on 85% of the dataset and tested on 15% according to Table 2. A stratified random split of 85:15 was applied, which means that both the model-building set and the testing set were divided in such a way that each split contained the same proportion of samples for each class as in the original dataset. MATLAB 2024a was employed to implement the architecture presented in Figure 8, and the following training options were used:  
* 8426 images, 90% for training and 10% for validation;
* shuffling every epoch;
* stochastic gradient descent with momentum (SGDM) optimizer;  
Figure 8: Common architecture of all four 2D-CNN-based models. * learning rate: 0.01;
* a mini-batch size (mbs) of 126 (60 iterations per epoch) for the models on one axis and mbs = 42 (180 iterations per epoch) for the model on all three axes;
* validation frequency: 30 iterations or 60 iterations;
* a maximum number of epochs of 60 for Mcwt_x, Mcwt_y, Mcwt_z models and 30 for Mcwt_xyz model;
* output/final model chosen based on the best validation loss criterion. All experiments were run on a computer with the processor AMD Ryzen 5 5600H, 3.3 GHz, with Radeon(TM) Graphics (2.0 GB), and Windows 10 operating system. In the training process of the CNN models, using MATLAB 2024a, 1 GPU was used. To assess the performance and the stability of the four proposed models (Mcwt_x, Mcwt_y, Mcwt_z, and Mcwt_xyz), 10 runs were performed for each model, with stratified random splits for the training and testing sets. In Figure 9, the accuracy and loss curves give insights into the learning process of the Mcwt_xyz model in the best-performing run (from 10 runs), with a testing accuracy of 100%. In the training process, 30 epochs (5400 iterations) and 90 validations were performed. Accuracy is an evaluation metric that expresses the overall performance of the model and is computed as the percentage of the number of correct predictions out of all tested instances. The loss metric measures the error (dissimilarity) between the predicted output and the true output. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline  & \multicolumn{5}{c}{**Number of Observations**} \\ \cline{2-6}  & \multirow{2}{*}{**Total**} & \multicolumn{5}{c}{**Classes**} \\ \cline{3-6}  & & **D1** & **D2** & **D3** & **D4** \\ \hline Dataset (100\%) & 9913 & 2593 & 2440 & 2440 & 2440 \\ \hline Training set (85\%) & 8426 & 2204 & 2074 & 2074 & 2074 \\ \hline Testing set (15\%) & 1487 & 389 & 366 & 366 & 366 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Dataset statistics. Figure 9: Learning and loss curves of Mcwt_xyz model in one run. The convergence in the learning process (Figure 9) is symbolized by the plateau (100% accuracy) reached in the training accuracy curve after 17 epochs. The accuracy curve for validation, following closely the curve for training, indicates that the model can generalize well to unseen data. The validation accuracy is between 99.8% and 100% after 20 epochs. The loss training curve shows a gradual decrease of the loss values during the performed iterations, resulting in an improvement of the model's performance. The losses measured during validations diminish too, so the model is making better predictions over time. The final (best) model is chosen based on the best validation loss criterion, and in Figure 9, we see that the best model is obtained in the 29 th epoch and the validation accuracy is 100%. After analyzing the learning curves for accuracy and loss in all 10 runs of Mcwt_xyz, we note that the model needs 10-11 epochs (15 min) to achieve 99% validation accuracy. After 20 epochs (30 min) the validation accuracy exceeds 99.8%, and in the next 10 epochs the model slightly improves and the best model is obtained. For each model, a global confusion matrix is calculated as the sum of the confusion matrices of the 10 runs performed. Based on such a matrix, mean performance metrics are calculated.