# Soft Sensor Transferability: A Survey
## 3 Transfer Learning  
As stated before, TL methodologies are useful when the data distribution of the target domain is different from the data distribution of the source domain [9]. Figure 2 shows the difference in the learning process in cases of traditional ML and TL. In the first case, the task is learned from scratch each time, while in the TL case the knowledge acquired from a previously learned source task is, in some way, transferred while learning a target task, to overcome scarcity in high-quality target data. A practical and common example is given by the problem of sentiment classification, which consists of a classification task on reviews of a specific type of product as a positive or negative view. To train the classifier, many reviews are first collected and labeled. When the type of product changes, such a classifier will not maintain good performance, if the distribution of data differs from the previous product. This means that new data should be  
Figure 2: Difference in the learning process in the case of traditional ML (**a**) and TL (**b**). collected and labeled for each type of product to create a new classifier. Such a procedure indeed being very expensive, TL techniques allow use of a classification model trained for some specific products to adapt to others of a different type, saving a great amount of effort [37]. Many other examples of TL are observable in nature as well, as shown in Figure 3.  
Humans themselves can intelligently apply the knowledge previously learned from one task to solve new ones faster or more efficiently. For instance, the knowledge acquired by learning a musical instrument would allow one to learn a new instrument faster. Formal definitions to introduce the TL problem will now be provided. Given a feature space \(\mathcal{X}\), an instance set \(X\) defined as \(X=\{x_{1},\ldots,x_{n}\}\in\mathcal{X}\) and its marginal probability distribution \(P(X)\), a _domain_\(\mathcal{D}\) is defined as  
\[\mathcal{D}=\{\mathcal{X},P(X)\}. \tag{5}\]  
This means that two domains differ when either their \(\mathcal{X}\) or \(P(X)\) differ. Given a domain \(\mathcal{D}\), a prediction, or a _task_, \(\mathcal{T}\) is defined by a label space \(\mathcal{Y}\) and a predictive function \(f(\cdot)\) as  
\[\mathcal{T}=\{\mathcal{Y},f(\cdot)\}. \tag{6}\]  
This again implies that two tasks differ when either their \(\mathcal{Y}\) or \(f(\cdot)\) differ. The predictive function \(f(\cdot)\) is not known a priori, but learned from the training data, which consist of the labeled pairs \(\{x_{i},y_{i}\}\), where \(x_{i}\in X,\ y_{i}\in\mathcal{Y}\), with \(i=1\ldots,n\). Given a new instance \(x\), then \(f(\cdot)\) can be used to predict its corresponding label \(f(x)\), that from a probabilistic point of view can be written as \(P(y|x)\). In the TL problem, distinction is made between a _source domain_\(\mathcal{D}_{\mathcal{S}}\) and its corresponding _source task_\(\mathcal{T}_{\mathcal{S}}\); a _target domain_\(\mathcal{D}_{\mathcal{T}}\) and its _target task_\(\mathcal{T}_{\mathcal{T}}\). The source domain \(\mathcal{D}_{\mathcal{S}}\) is usually observed via the instance-label pairs as  
\[D_{\mathcal{S}}=\Big{\{}\big{(}x_{\mathcal{S}_{1}},y_{\mathcal{S}_{1}}\big{)} \ldots,\Big{(}x_{S_{\mathcal{S}_{\mathcal{S}_{\mathcal{S}_{\mathcal{S}_{ \mathcal{S}_{\mathcal{S}_{\mathcal{S}}_{\mathcal{S}}_{\mathcal{S}}_{\mathcal{S }}}}}}}}}\Big{)}\Big{\}},\ \text{where}\ x_{\mathcal{S}_{i}}\in\mathcal{X}_{\mathcal{S}}\ \text{and}\ y_{\mathcal{S}_{i}}\in \mathcal{Y}_{\mathcal{S}}. \tag{7}\]  
Whereas, the target domain data are denoted as  
Figure 3: Examples of human intuitive transfer learning: music instruments (**a**), board games (chinese chess and chess) (**b**), two-wheeled motor vehicles (**c**).