# Soft Sensor Transferability: A Survey
## 3 Transfer Learning
### Distribution Distance Metrics  
Metrics to evaluate the differences among data distributions are commonly adopted in feature-based TL to learn a new space that reduces the difference of distribution between the two domains. How to measure the distribution difference or the similarity between domains is, therefore, an important task. They are used in instance-based TL methods as well to produce the weights of the instances by minimizing the adopted metric between the domains [40]. In Table 2 the most adopted metrics in TL techniques are reported. They are defined in the following. * **Maximum Mean Discrepancy** (MMD) [41] Given two distributions \(P\) and \(Q\), MMD is defined as the distance between the means of them mapped into a Reproducing Kernel Hilbert Space (RKHS): \[MMD(P,Q)=\parallel\mu_{P}-\mu_{Q}\parallel_{\mathcal{H}}.\] (11) where \(\mu\) represents the mean value of the distribution. The MMD is one of the most used measures in TL. One known feature representation method for TL called _Transfer Component Analysis_ (TCA) [42] learns some transfer components across domains in an RKHS using MMD. Another unsupervised feature transformation technique called _Joint Distribution Adaptation_ (JDA) jointly adapts both the marginal and conditional distributions of the domains in a dimensionality reduction procedure based on Principal Component Analysis (PCA) and the MMD measure [43]. \begin{table}
\begin{tabular}{l l l} \hline \hline
**Distribution Difference Measure** & **Algorithm** & **Applications** \\ \hline Maximum Mean Discrepancy (MMD) & [41] & [42; 43] \\ Kullback–Leibler Divergence (\(D_{KL}\)) & [44] & [45; 46] \\ Jensen–Shannon Divergence (JSD) & [47] & [48; 49] \\ Bregman Distance (\(D_{F}\)) & [50] & [51; 52] \\ Hilbert–Schmidt Independence Criterion (HSIC) & [53] & [54; 55] \\ Wasserstein Distance (W) & [56] & [57; 58] \\ Central Moment Discrepancy (CMD) & [59] & [60; 61] \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Metrics commonly adopted in TL to quantify the distribution difference between datasets. * **Kullback-Leibler Divergence** (\(D_{KL}\)) [44] \(D_{KL}\) is an asymmetric measure of how one probability distribution differs from another. Given two discrete probability distributions, \(P\) and \(Q\) on the same probability space \(\mathcal{X}\), \(D_{KL}\), or the relative entropy, from \(Q\) to \(P\) is defined as: \[D_{KL}(P\parallel Q)=\sum_{x\in\mathcal{X}}P(x)log\bigg{(}\frac{P(x)}{Q(x)} \bigg{)}.\] (12) In [PERSON] et al. [45] a supervised representation learning method based on deep autoencoders for TL is introduced so that the distance in distributions of the instances between the source and the target domains is minimized in terms of \(D_{KL}\). Feature-based TL realized through autoencoders is proposed in [PERSON] et al. [46], where \(D_{KL}\) is adopted to measure the similarity of new samples concerning historical data samples. * **Jensen-Shannon Divergence** (JSD) [47; 48; 49] \(\quad\) JSD is a symmetric and smooth version of \(D_{KL}\), defined as: \[JSD(P\parallel Q)=\frac{1}{2}D_{KL}(P\parallel M)+\frac{1}{2}D_{KL}(Q \parallel M),\] (13) with M being \(M=\frac{1}{2}(P+Q)\). JSD is used in [PERSON] et al. [48] as a distance metric in a clustering technique for domain adaptation purposes. A classification method for TL proposed in [PERSON] et al. [49] exploits the JSD measure with a PCA feature mapping technique. * **Bregman Distance** (\(D_{F}\)) [50] \(D_{F}\) is a difference measure between two points defined in terms of a strictly convex function called _Bregman function_\(F\). The points can be interpreted as probability distributions.