# Deep Transfer Learning-Based Fault Diagnosis Using Wavelet Transform for Limited Data
## 5 Performance Evaluation
### Diagnosis Performance Analysis  
Figure 7 shows the confusion matrices of three classification models, namely, SVM, the CNN, and CNN-TL with \(r=0.3\). It summarizes the fault diagnosis results of each fault type, including the healthy state \(f_{0}\) and faulty states \(\{f_{1},\ldots,f_{7}\}\). We set the severity level \(l\geq 1\) as the fault condition of each fault type. The longitudinal and transverse axes present the true and predicted labels, respectively. The diagonal entries indicate the number of correctly predicted fault types, while the off-diagonal entries are the number of incorrectly predicted fault types. A normalized row value (resp. normalized column value) shows the percentages of correctly and incorrectly classified observations for each true class (resp. each predicted label). Overall, the classification accuracy of the CNN-TL is 0.96, which is higher than 0.65 and 0.82 of SVM and the CNN, respectively. Figure 6: Wavelet images of different fault types, \(f_{0},\ldots,f_{7}\). One interesting observation is that the confusion matrix has the form of the tridiagonal matrix, and the off-diagonal entries are not fully spread. It means most classification errors tend to be more pairwise than independent random errors. Let us first consider the healthy state \(f_{0}\) and the blocked inlet fault \(f_{2}\). In the confusion matrix, some faults between \(f_{0}\) and \(f_{2}\) using SVM are incorrectly classified, while the corresponding errors are very low for the CNN and CNN-TL. In Figure 4, we have shown that the healthy state \(f_{0}\) and the blocked inlet fault \(f_{2}\) have similar characteristics in both the time and frequency domains. However, both the CNN and CNN-TL models efficiently handle the classification between \(f_{0}\) and \(f_{2}\). Now, we consider the pair of \((f_{1},f_{3})\) for which most errors of both \(f_{1}\) and \(f_{3}\) are isolated from other fault types. SVM erroneously classifies the cylinder leak fault \(f_{1}\) to the combined cylinder leak and blocked inlet fault \(f_{3}\) when the effect of the blocked inlet is not severe. We also observe similar classification errors of \(f_{3}\) to \(f_{1}\) for the SVM model. Remember that we observe similar wavelet images between \(f_{1}\) and \(f_{3}\) in terms of the positions and the number of horizontal stripes in Figure 6. Both the CNN and CNN-TL reduce the classification errors of \(f_{1}\) to \(f_{3}\), while the CNN model still has the low classification accuracy of \(f_{3}\) due to the large number of classification errors of \(f_{1}\), similar to that of SVM. Thus, the classification errors are asymmetric even in the same pair of \((f_{1},f_{3})\) for the CNN and CNN-TL. One of the major classification errors is related to the bearing friction fault \(f_{4}\), namely, \(f_{5},f_{6}\), and \(f_{7}\). These pairwise errors are separated from \(f_{0},\ldots,f_{3}\) since the corresponding part of the confusion matrix has the form of a block matrix. The two dominant pairs of classification errors are \((f_{4},f_{6})\) and \((f_{5},f_{7})\). The cylinder leak fault makes a noticeable difference between these two pairs, as shown in Figure 6. The classification errors in each pair of \((f_{4},f_{6})\) and \((f_{5},f_{7})\) considerably depend on the severity level of the blocked inlet. The CNN-TL still performs better than other models for these pairs. The average accuracy related to \(f_{4}\) of SVM, the CNN, and CNN-TL are \(0.54,0.81\), and \(0.97\), respectively. Next, we investigate the performances of the three classification models when limited samples with different severity levels are adopted to train the network. Thus, the ratio of training target samples varies from \(0.1\) to \(0.8\). Note that we fix the ratio of the validation and testing data set as \(0.1\). Figure 8 shows the classification accuracy of SVM, the CNN, and CNN-TL with different severity levels \(l\geq 1,l=1,6\) as a function of various ratios of training data.