# End-To-End Convolutional Neural Network Model for Fear Fault Diagnosis Based on Sound Signals
## 4 Experiment Result and Analysis
### Evaluation  
End-to-end stacked CNN models were trained with the cross-entropy loss for multi-faulty type classification. An Adam optimization algorithm was employed as the training algorithm. The learning  
Figure 8: Waterfall diagrams of four different types of gears under two-load conditions. (**A**) Represents a no-load condition and (**B**) represents a load condition. Figure 9: Time and frequency domain signal from four microphones. rate was 0.003. In addition, we used the early stopping approach to prevent overfitting and adopted the no-improvement-in-10-epochs strategy to identify the number of epochs via the validation set. Finally, our models adopted 70 and 50 of training epochs on datasets A and B, respectively. In order to show the superiority of our method, we compared our model with traditional methods that combined manual features with conventional machine learning algorithms. The manual features that we extracted include a log-mel spectrogram, root-mean-square error (RMSE), and a spectral centroid. We combined these manual features as a matrix and fused multi-channel signals from four different microphones at the feature level. Four evaluation methods were designed to evaluate the generalization capability of the model:  
In Evaluation 1, we trained a CNN model using the training samples from dataset A and tested it on the test samples in dataset A. In Evaluation 2, we trained a CNN model using the training samples from dataset B and tested it on the test samples in dataset B. In Evaluation 3, we trained a CNN model using the training samples from dataset A and tested it on the test samples in dataset B. In Evaluation 4, we trained a CNN model using the training samples from dataset B and tested it on the test samples in dataset A. We used the classification accuracy as the evaluation criterion, which is defined as \(acc=\frac{N_{1}}{N_{2}}\), where \(N_{1}\) represents the number of test samples recognized properly, and \(N_{2}\) represents the total number of test samples. The test results on two datasets are shown in Table 3.  
From Table 3, we can see that the recognition accuracy of our end-to-end model reached 100% in Evaluation 1 and Evaluation 2 respectively. It was 1.2% and 1.4% higher than the accuracy of the best traditional method: manual features + SVM. In order to further evaluate the generalization capability of our model, we added two more evaluation methods. The first method was training the model using the training samples in dataset A and testing it on test samples in dataset B. The second way was exchanging the training set and testing set in the first method. The reason why we adopted these evaluation methods was that a fault diagnosis algorithm with good generalization capabilities should still work even when the load condition changes. The test result showed that all methods decline in recognition accuracy, especially RandomForest and GBDT with manual features. The accuracy of RandomForest declined from 97.5% when evaluated on dataset A without load to 60.3% when tested on dataset B with a load with the same trained model. Similarly, the accuracy of GBDT declined from 95.7% when evaluated on dataset A without load to 58.9% when tested on dataset B with a load with the same trained model. Conversely, when we trained the RandomForest and GBDT models on dataset A without load and tested them on dataset B with a load, the prediction accuracy declines (from 93.6% to 87.7% for RandomForest; 96.5% to 93% for GBDT) were much smaller compared the previous cases. We suspect that some relationship between the feature spectrum and fault types exist in dataset B but not in dataset A so that training