# Soft Sensor Transferability: A Survey
## 3 Transfer Learning
### Distribution Distance Metrics
Given \(F:\Omega\rightarrow\mathbb{R}\) a continuously-differentiable, strictly convex function defined on a closed convex set \(\Omega\), the Bregman distance \(D_{F}\) associated with F for points \(p,q\in\Omega\) is defined as the difference between the value of \(F\) at point \(p\) and the value of the first-order Taylor expansion of \(F\) around point \(q\) evaluated at point \(p\): \[D_{F}(p,q)=F(p)-F(q)-\langle\triangledown F(q),p-q\rangle.\] (14) A TL method for hyperspectral image classification proposed in [PERSON] et al. [51] employs a regularization based on \(D_{F}\) to find common feature representation for both the source domain and target domain. A domain adaptation approach introduced in [PERSON] et al. [52] reduces the discrepancy between the source domain and the target domain in a latent discriminative subspace by minimizing a \(D_{F}\) matrix divergence function. * **Hilbert-Schmidt Independence Criterion** (HSIC) [53] Given separable RKHSs \(\mathcal{F}\), \(\mathcal{G}\) and a joint measure \(p_{xy}\) over \((\mathcal{X}\times\mathcal{Y},\Gamma\times\Lambda)\), HSIC is defined as the squared HS-norm of the associated cross-covariance operator \(C_{xy}\): \[HSIC(p_{xy},\mathcal{F},\mathcal{G}):=\ \ \parallel C_{xy}\parallel_{HS}^{2}.\] (15) A domain adaptation method called _Maximum Independence Domain Adaptation_ (MIDA) finds a latent feature space in which the samples and their domain features are maximally independent in the sense of HSIC [54]. Another method to find the structural similarity between two source and target domains is proposed in [PERSON] and [PERSON] [55]. The algorithm extracts the structural features within each domain and then maps them into the RKHS. The dependencies estimations across domains are performed using the HSIC. * **Wasserstein Distance** (W) [56] Given two distributions \(P\) and \(Q\), the \(p\)th Wasserstein distance metric W is defined as: \[W:=W(F_{P},F_{Q})=\bigg{(}\int_{0}^{1}\mid F_{P}^{-1}(u)-F_{Q}^{-1}(u)\mid^{p} du\bigg{)}^{\frac{1}{p}},\] (16)where \(F_{P}\) and \(F_{Q}\) are the corresponding cumulative distribution functions and \(F_{P}^{-1}\) and \(F_{Q}^{-1}\) the respective quantile functions. \(W\) is employed in [PERSON] et al. [57] for an algorithm that aims to learn domain invariant feature representation. It utilizes an ANN to estimate the empirical \(W\) distance between the source and target samples and optimizes a feature extractor network to minimize the estimated \(W\) in an adversarial manner. A \(W\)-based asymmetric adversarial domain adaptation is proposed also in [PERSON] et al. [58] for unsupervised domain adaptation for fault diagnosis. * **Central Moment Discrepancy (CMD)**[59] CMD is a distance function on probability distributions on compact intervals. Given two bounded random vectors \(X=(X_{1},\ldots,X_{N})\) and \(Y=(Y_{1},\ldots,Y_{N})\) i.i.d. and two probability distributions \(P\) and \(Q\) on the compact interval \([a,b]^{N}\), CMD is defined as \[CMD(P,Q)=\frac{1}{\mid b-a\mid}\parallel\mathbb{E}(X)-\mathbb{E}(Y)\parallel_{ 2}+\sum_{k=2}^{\infty}\frac{1}{\mid b-a\mid^{k}}\parallel\hat{c}_{k}(X)-c_{k}( Y)\parallel_{2},\] (17) where \(\mathbb{E}(X)\) is the expectation of \(X\) and \(c_{k}(X)\) is the central moment vector of order \(k\) defined in [PERSON] et al. [59]. In a domain adaptation method for fault detection presented in [PERSON] et al. [60], a CNN is applied to extract features from two differently distributed domains and the distribution discrepancy is reduced using the CMD criterion. Another CNN- and CMD-based for fault detection is proposed in [PERSON] et al. [61]. Since it is often difficult to design metrics that are well-suited to the particular data and task of interest, an ML field called _Distance Metric Learning_ (DML) aims at automatically constructing task-specific distance metrics from supervised data [62]. As an ML task, DML suffers the same problems described so far, requiring a large amount of label information.