# Soft Sensor Transferability: A Survey
## 3 Transfer Learning
spaces are the same \(\mathcal{X_{S}}=\mathcal{X_{T}}\), but the marginal probability distributions of the inputs differ \(P(X_{\mathcal{S}})\
eq P(X_{\mathcal{T}})\), such TL setting is referred to as _domain adaptation_[39];
* **Unsupervised TL:** target and source tasks are different and the label information is unknown for both the source and the target domains. This means that by definition such setting regards clustering and dimensionality reduction tasks, and not classification or regression as in the previous cases. For this reason, given the application of SSs, unsupervised solutions are not considered. Another categorization is based on the consistency between the feature and label spaces from the source and the target. * **Homogeneous TL:** if \(\mathcal{X_{S}}=\mathcal{X_{T}}\) and/or \(\mathcal{Y_{S}}=\mathcal{Y_{T}}\);
* **Heterogeneous TL:** if \(\mathcal{X_{S}}\
eq\mathcal{X_{T}}\) and/or \(\mathcal{Y_{S}}\
eq\mathcal{Y_{T}}\). Besides the label settings or the consistency of the spaces, TL techniques can be categorized based on \"what\" is transferred, leading to four groups: _instance-based_, _feature-based_, _parameter-based_ and _relational-based_ (see Figure 5). * **Instance-based transfer:** these approaches assume that due to the difference in distributions between the source and the target domains, certain parts of the data in the source domain can be reused, by _reweighting_ them so as to reduce the effect of the \"harmful\" source data, while encouraging the \"good\" data;
* **Feature-based transfer**: the idea behind this approach is to learn a \"good\" feature representation for the target domain, to minimize the marginal and the conditional distribution differences, preserving the properties or the potential structures of the data, and classification or regression model error. For instance, a solution is to find  
Figure 4: Transfer learning technique categorization under a label-setting view. Source input (\(X_{S}\)), source output (\(Y_{S}\)), target input (\(X_{T}\)) and target output (\(Y_{T}\)) are shown. Figure 5: Transfer learning techniques categorization under a “what” is the transferred view. the common latent features through feature transformation and use them as a bridge to transfer knowledge: this case is referred to as a _symmetric feature-based transfer_ since both the source and the target features are transformed into a new feature representation; in contrast, in the _asymmetric_ case, the source features are transformed to match the target ones. When performing feature transformation to reduce the distribution difference, one issue is how to measure such differences or similarities. This is done through specific ad-hoc metrics, which are described in Section 3.1;
* **Parameter-based transfer**: this approach performs the transfer at the model/parameter level by assuming that models for related tasks should share some parameters or prior distributions of hyperparameters. So, by discovering them, knowledge is transferred across tasks themselves;
* **Relational-based transfer**: these approaches deal with transfer learning for relational domains, where the data are non-independent and identically distributed (i.i.d.) and can be represented by multiple relations. The assumption is that some relationship among the data in the source and target domains is similar and that is the knowledge to be transferred, transferring the logical relationship or rules learned in the source domain to the target domain. The references reported in this paper are classified based on the above categories. In the next section, some metrics generally used in the TL framework are briefly introduced.