# Intra-Domain Transfer Learning for Fault Diagnosis with Small Samples
## 2 Methodology
### Intra-Domain Transfer Learning Strategy  
Transfer learning refers to the application of experiences gained in solving one task to a different but similar task. Experiences learned from the source domain are used as prior knowledge to solve target problems. Put formally, given a source domain \(D_{S}\) and a target domain \(D_{T}\), transfer learning reuses the information embedded in \(D_{S}\) for better problem-solving in \(D_{T}\). Through knowledge sharing, the requirements for supervisory instances in \(D_{T}\) can be greatly reduced, thus making it possible to learn even when samples are small.  
The above statement holds true in general when the source and target domains are closely related. However, if there is a significant covariate shift between \(D_{S}\) and \(D_{T}\), transfer learning might inversely hurt the target performance, i.e., negative transfer [22; 26]; see Figure 1. In other words, inter-domain transfer may take the learning in the wrong direction in the target domain. To mitigate the impact of negative transfer, researchers have proposed many workarounds, including instance weighting [29], feature matching [30], and transitive transfer [31]. Inspired by the Distant Domain Transfer Learning (DDTL) approach in [26] where domain discrepancies are substantial, we propose an intra-domain transfer learning strategy.  
The intra-domain transfer learning strategy assumes the source domain is different from but akin to the target domain. Even so, negative transfer can still occur. To preclude this, we propose a two-stage transfer scheme: first, we construct a pre-trained model in the source domain via a vanilla transfer from an off-the-shelf inter-domain deep neural network; then, we transfer the pretrained model to the target domain via shallow-layer freezing and finetuning. Figure 2 depicts the two-stage intra-domain transfer learning strategy.  
In our problem setting, the source domain is data-abundant, and this prevents negative transfer, even though we use an inter-domain vanilla transfer. We also use the first-stage  
Figure 1: Illustration of negative transfer. The two oval dashed lines represent the distribution shape in the source and target domains. Transfer learning facilitates the training of a finetuned classifier in the target domain even with small samples, but distribution discrepancy may cause negative transfer, leading to poor generalization in target tasks.  
transfer to speed up model training in the source domain. In the second stage, the shallow-layer freezing is intended to ensure knowledge transfer between akin domains, while finetuning uses the limited number of samples in the target domain to further train the model for better performance in target tasks.  
In addition to requiring relative proximity between source and target domains, the intra-domain transfer learning strategy requires the source domain to be data-abundant. Note that in the field of fault diagnosis, annotated faulty samples are less expensive to acquire in the laboratory than in real-world applications. Therefore, a deluge of labelled samples in the source domain can be generated via fault injection experiments or simply first-principal simulation. This constitutes another motivation of this research.