## 5 Experiment Result and Comparative Study
### Comparison of Different Methods  
In this part, to further illustrate that the proposed method has high diagnostic performance, the four different evaluation indexes of the proposed method are compared with those of other methods. Evaluation indicators include average _Accuracy_, _Precision_, _Recall_ and _F1-score_. The formulae for average _Accuracy_, _Precision_, _Recall_ and _F1-score_ are (18)-(21). \[Accuracy=\frac{|TP|+|TN|}{|TP|+|FP|+|TN|+|FN|} \tag{18}\]  
\[Precision=\frac{|TP|}{|TP|+|FP|} \tag{19}\]  
\[Recall=\frac{|TP|}{|TP|+|FN|} \tag{20}\]  
\[F1\text{-}score=\frac{2*Precision*Recall}{Precision+Recall} \tag{21}\]  
Take #0 as an example: \(|TP|\) represents the number of samples classified correctly into #0, \(|TN|\) represents the number of samples classified correctly into other defects, \(|FP|\) represents the number of samples classified incorrectly into #0 and \(|FN|\) represents the numbers of samples in #0 classified into other classes. The _F1-score_ is used to reflect the overall indicator comprehensively. Here are six different comparison methods:  
(1) Artificial neural network (ANN) using raw vibration signals (RVS) as input (RVS + ANN);  
(2) ANN using HHT time-frequency images as input (HHT + ANN);  
(3) ANN using CWT time-frequency images as input (CWT + ANN);  
(4) One\(-\)dimensional CNN using RVS as input (RVS + 1 DCNN);  
(5) CNN using HHT time-frequency images as input (HHT + CNN);  
(6) CNN using CWT time-frequency images as input (CWT + CNN);  
The input shapes and parameters of the other methods are shown in Table 3. Different methods record ten trials of diagnosis accuracy, and the average accuracy is utilized to quantify the method, as shown in Table 3. The other evaluation indicators of the different methods are shown in Figure 10 for each type of manufacturing defect. Figure 9: Visualization maps for the learned characteristics. raw vibration signal for manufacturing defect diagnosis. The training time of the proposed method is only lower than that of the HHT + ANN method and CWT + ANN method. The testing time of each sample of the proposed method is larger than that of other methods, but the test rate meets the requirements of the production line. Figure 10 clearly shows that the evaluation indicator of the proposed method is generally better than that of the other methods for each typical manufacturing defect. A detailed analysis of #0 shows that the recall of CWT + ANN is 9.9% higher than that of the proposed method. However, the precision of CWT + ANN is 31.2% lower than that of the proposed method in #0. From the analysis of CWT + ANN, it can be observed that a small number of normal compressor (#0) samples are misdiagnosed as other defects, but some defective compressor samples are misdiagnosed as #0. If the faulty samples are misdiagnosed as normal samples, it may lead to the failure of the refrigeration system. Thus, it is unreliable to rely only on recall and precision to evaluate the model. The F1-score is an evaluation index that combines the two indicators and comprehensively reflects the overall indicator. It can be seen from Figure 10c that the F1-score of the proposed method is 13.6% higher than that of the CWT + ANN for #0. A detailed analysis of #3 and #5 reveals that recall, precision and F1-score of the proposed method are higher than that of other methods. The recall, precision and F1-score of each typical manufacturing defect are mostly greater than 90%. It can be proved that the performance of the proposed method is superior to other methods for the diagnosis of compressor manufacturing defects. Since the dual time-frequency image features extracted can be complementary, the proposed methods obtain richer features and higher diagnostic performance. The hermetic refrigeration compressor usually is tested in varying ambient noise levels. Therefore, it is necessary to study the diagnosis performance of the proposed methods under various conditions with a low signal-to-noise ratio (\(SNR\)).