# Soft Sensor Transferability: A Survey
## 3 Transfer Learning
\[D_{\mathcal{T}}=\Big{\{}\big{(}x_{\mathcal{T}_{1}},y_{\mathcal{T}_{1}}\big{)}, \ldots,\big{(}x_{\mathcal{T}_{n\mathcal{T}}},y_{\mathcal{T}_{n\mathcal{T}}} \big{)}\Big{\}},\ \text{where}\ x_{\mathcal{T}_{i}}\in\mathcal{X}_{\mathcal{T}}\ \text{and}\ y_{\mathcal{T}_{i}}\in \mathcal{Y}_{\mathcal{T}}. \tag{8}\]  
In most real applications, an observation of the target domain consists of unlabeled instances or just a limited number of labeled ones, meaning that usually \(0\leq n_{\mathcal{T}}\ll n_{\mathcal{S}}\). TL aims to improve the learning of the target predictive function \(f_{\mathcal{T}}(\cdot)\) in \(\mathcal{D}_{\mathcal{T}}\) using the knowledge in \(\mathcal{D}_{\mathcal{S}}\) and \(\mathcal{T}_{\mathcal{S}}\), where \(\mathcal{D}_{\mathcal{S}}\
eq\mathcal{D}_{\mathcal{T}}\), \(\mathcal{T}_{\mathcal{S}}\
eq\mathcal{T}_{\mathcal{T}}\). When the target and source domains are the same, \(\mathcal{D}_{\mathcal{S}}=\mathcal{D}_{\mathcal{T}}\), and their tasks are the same, \(\mathcal{T}_{\mathcal{S}}=\mathcal{T}_{\mathcal{T}}\), then it becomes an usual ML problem. As already stated, the condition \(\mathcal{D}_{\mathcal{S}}\
eq\mathcal{D}_{\mathcal{T}}\) implies that  
\[\mathcal{X}_{\mathcal{S}}\
eq\mathcal{X}_{\mathcal{T}}\ \text{or}\ P(X_{\mathcal{S}})\
eq P(X_{\mathcal{T}}). \tag{9}\]  
Analogously, \(\mathcal{T}\) being defined as \(\mathcal{T}=\{\mathcal{Y},P(Y|X)\}\), the condition \(\mathcal{T}_{\mathcal{S}}\
eq\mathcal{T}_{\mathcal{T}}\) means that  
\[\mathcal{Y}_{\mathcal{S}}\
eq\mathcal{Y}_{\mathcal{T}}\ \text{or}\ P(Y_{\mathcal{S}}|X_{\mathcal{S}})\
eq P(Y_{\mathcal{T}}|X_{ \mathcal{T}}). \tag{10}\]  
Finally, when there exists some kind of relationship between the feature spaces of the two domains, then the domains are said to be _related_. In some cases, when the two domains are not related, a knowledge transfer could be unsuccessful at the point of worsening the learning in the target domain. When the target learner is indeed hurt by the transfer, the phenomenon is referred to as _negative transfer_[38]. The above definitions allow performance a categorization of TL techniques. Approaches can be grouped on a different basis, in particular under a _problem_ point of view and an _approach_ point of view. In the first case, the categorization can be performed on either the presence of labels in the source and target datasets (_label setting_) or the consistency of the feature space (_space setting_); the latter categorizes the types of TL techniques based on \"what\" part of the knowledge from the source is actually transferred to the target. The classification is reported in Table 1 and described in detail as follows. From the label-setting aspect, TL techniques can be categorized into three types, based on the possible different situations between domains and tasks of the source and the target: inductive transfer learning, transductive transfer learning, unsupervised transfer learning (see Figure 4). * **Inductive TL**: target and source tasks are different, regardless of the domains, and the label information of the target domain instances is available. Target-labeled data induce the learning of the target predictive model, hence the name;
* **Transductive TL**: target and source domains differ and the label information only comes from the source domain. In this case, if the domains differ because the feature \begin{table}
\begin{tabular}{c c c} \hline \hline
**TL Categories** & & **Types** \\ \hline \multirow{4}{*}{Problem categorization} & \multirow{2}{*}{Label setting} & Inductive \\  & & Transductive \\  & & Unsupervised \\ \cline{2-3}  & & Space setting \\ \hline \multirow{4}{*}{Approach categorization} & \multirow{4}{*}{“What” is transferred} & Instance-based \\  & & Feature-based \\ \cline{1-1}  & & Parameter-based \\ \cline{1-1}  & & Relational-based \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Most commonly adopted categorizations of TL techniques, under a “problem” point of view and an “approach” point of view.