# Intra-Domain Transfer Learning for Fault Diagnosis with Small Samples
## 4 Demystifying the \"Black-Box\" via Visualization
### Qualitative Assessment of the Heat Maps of the Target Task  
The ability to learn high-level abstraction of features using CNN has already been verified in computer vision applications but is relatively underexplored in fault diagnosis. Armed with the above heat map method, we attempt to visualize those high-level features extracted by the transferred ResNet-50 model in our target task and correlate them with the underlying fault characteristics. Hereinafter, to validate the efficacy of our proposed intra-domain transfer learning strategy using small samples, we analyze the model finetuned when the splitting ratio of training and testing samples is set at 1:99.  
Figure 11 shows some heat maps generated by the fault diagnosis model at testing time. Each heat map corresponds to one random testing sample in a specific working condition in our target task; see Table 1 for a reference. Three frequency spectrums are presented in Figure 11. Healthy samples are normally characterized by an amplitude located at the shaft rotating frequency and its harmonies, as shown in Figure 11a, while faulty samples can be recognized by anomalous amplitude in their characteristic frequencies.  
The activations highlighted in red boxes are consistent with their corresponding fault characteristics. For example, as shown in the frequency spectrum of Figure 11c, the fault \"crack\" has a characteristic frequency around 5000 Hz, and the energy in this frequency band is correctly identified in its heat map. Similar results can be observed in other fault classes. Notably, the horizontal axis in our heat map is time, the same as in the time-frequency image. Therefore, the horizontal location of the most activated area can also signify the temporal features of non-stationary signals that lead to CNN's final classification.  
Another observation is the most activated spots in these heat maps are distinctive, making it easy to separate them. Figure 12 compares the heat maps of the various working conditions obtained via the three learning schemes. Intuitively, it can be deduced that TFC leads to the highest differentiability in these heat maps, TFI ranks second, and TFS comes in last. This coincides with the confusion matrices shown in Figure 9 and the testing accuracy results in Table 3.  
From a result-oriented perspective, higher differentiability in the heat maps would yield higher diagnostic accuracy. But it by no means implies that the most activated area in the feature maps must match the underlying physical mechanisms of the corresponding faults. The above qualitative assessment of the heat maps not only proves the efficacy of the proposed intra-domain learning strategy, but also discloses how the transferred ResNet-50 model infers the diagnostic results. Since the feature extractors learned from an intra-domain task are frozen in our target-domain task, the knowledge sharing enabled by our transfer learning strategy is validated in the heat maps. Moreover, the heat maps of misclassification samples can provide new perspectives to improve the fault diagnosis model.  
Figure 10: An exemplar illustration of heat map visualization: (**a**) input image, (**b**) average activation matrix, (**c**) heat map.  
Figure 11: Heat maps and frequency spectrums of various states: (**a**) Healthy; (**b**) Spall; (**c**) Crack; (**d**) Chip.  
Figure 12: Heat map comparison of the three learning schemes under various working conditions.