# An Information Theoretic Approach to Privacy-Preserving Interpretable and Transferable Learning
## 5 Information Theoretic Measures for Privacy Leakage, Interpretability, and Transferability
### A Unified Approach to Privacy-Preserving Interpretable and Transferable Learning  
The presented theory allows us to develop an algorithm that implements privacy-preserving interpretable and transferable learning methodology in a unified manner. \begin{table}
\begin{tabular}{c c} \hline \hline
**Symbol/Mapping** & **Definition/Meaning** \\ \hline \(f_{y_{gz}\to c}:\mathbb{R}^{p_{gz}}\to\{1,\cdots,C\}\) & \begin{tabular}{c} mapping assigning class label to target domain data vector \(y_{tg}\) via (47), i.e., \\ \(f_{y_{gz}\to c}(y_{tg})=\ell\Big{(}y_{tg\to sr}(y_{tg});\{\mathcal{P}_{c}^{(g )}\}_{c=1}^{C},\{\mathcal{P}_{c}^{+sr}\}_{c=1}^{C},\mathbb{M}^{sr\to tg} \Big{)}\) \\ \end{tabular} \\ \hline \multirow{4}{*}{\(\hat{g}_{tg}^{sr}\in\mathbb{R}^{p_{sr}}\)} & \begin{tabular}{c} transformation of \(y_{tg}\) to source domain and filtering through the autoencoder \\ that represents the source domain feature vectors of the same class as that of \(y_{tg}\), i.e., \\ \(\hat{g}_{tg}^{sr}=\widehat{\mathcal{W}\mathcal{D}}\bigg{(}y_{tg\to sr}(y_{tg}); \mathcal{P}_{f_{y_{rg}\to c}(y_{tg})}^{+sr}\bigg{)}\) \\ \end{tabular} \\ \hline \multirow{4}{*}{\(\hat{g}_{tg}^{lg}\in\mathbb{R}^{p_{sr}}\)} & \
\begin{tabular}{c} transformation of \(y_{tg}\) to source domain and filtering through the autoencoder \\ that represents the target domain feature vectors of the same class as that of \(y_{tg}\), i.e., \\ \(\hat{g}_{tg}^{lg}=\widehat{\mathcal{W}\mathcal{D}}\bigg{(}y_{tg\to sr}(y_{tg}); \mathcal{P}_{f_{y_{rg}\to c}(y_{tg})}^{IS}\bigg{)}\) \\ \end{tabular} \\ \hline \multirow{4}{*}{\(f_{g_{tg}^{sr}\to g_{tg}^{lg}}:\mathbb{R}^{p_{sr}}\to\mathbb{R}^{p_{sr}}\)} &
\begin{tabular}{c} mapping from source domain feature vector \(\hat{g}_{tg}^{sr}\) to target domain feature vector \(\hat{g}_{tg}^{lg}\), i.e., \\ \(\hat{g}_{tg}^{lg}=f_{g_{tg}^{sr}\to g_{tg}^{lg}}\Big{(}\hat{g}_{tg}^{sr}\Big{)}\) \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 2: _Cont._Algorithm 9 is presented for a systematic implementation of the proposed privacy-preserving interpretable and transferable deep learning methodology. The functionality of Algorithm 9 is as follows:  
```
0: The labeled source dataset: \(\mathbf{Y}^{\pi r}=\{\mathbf{Y}_{c}^{\mathrm{c}}\}_{c=1}^{\mathrm{C}}\) (where \(\mathbf{X}_{c}^{\pi r}=\{y_{sr}^{\mathrm{c}}\in\mathbb{R}^{p_{sr}}\mid i\in\{1, \cdots,N_{c}^{\mathrm{c}}\}\}\)) represents the \(c\)-th labeled samples); the set of private data: \(\mathbf{X}^{\pi r}=\{\mathbf{X}_{c}^{\mathrm{cr}}\}_{c=1}^{\mathrm{C}}\) (where \(\mathbf{X}_{c}^{\pi r}=\{x_{sr}\in\mathbb{R}^{n_{sr}}\mid x_{sr}=f_{\sum_{s=y_{sr }}^{-1}}^{-1}(y_{sr}),\ y_{sr}\in\mathbf{Y}_{c}^{\pi r}\}\)); the set of interpretable parameters: \(\mathbf{T}^{\pi r}=\{\mathbf{T}_{c}^{\pi r}\}_{c=1}^{\mathrm{C}}\) (where \(\mathbf{T}_{c}^{\pi r}=\{t_{tr}\in\mathbf{T}^{\mathrm{c}}\mid t_{tr}=f_{\sum_{s =y_{sr}}^{-1}}^{-1}(y_{sr}),\ y_{sr}\in\mathbf{Y}_{c}^{\pi r}\}\)); the set of a few labeled target samples: \(\{\mathbf{Y}_{c}^{\mathrm{g}}\}_{c=1}^{\mathrm{C}}\) (where \(\mathbf{Y}_{c}^{\mathrm{g}}=\{y_{sr}^{\mathrm{fg}}\in\mathbb{R}^{p_{sr}}\mid i \in\{1,\cdots,N_{c}^{\mathrm{fg}}\}\}\)) is the set of \(c\)-th labeled target samples); the set of unlabeled target samples: \(\mathbf{Y}_{*}^{\mathrm{fg}}=\{y_{rg}^{\mathrm{fg}}\in\mathbb{R}^{p_{sr}}\mid i \in\{1,\cdots,N_{c}^{\mathrm{fg}}\}\}\); and the differential privacy parameters: \(d\in\mathbb{R}_{+},e\in\mathbb{R}_{+},\delta\in(0,1)\). 1: A differentially private approximation of source dataset, \(\mathbf{Y}^{+\pi r}=\{\mathbf{Y}_{c}^{\mathrm{g}+\pi}\}_{c=1}^{\mathrm{C}}\); is obtained using Algorithm 5 on \(\mathbf{Y}^{\pi r}\). 2: Differentially private source domain classifier, \(\{\mathcal{P}_{c}^{\mathrm{g}+\pi}\}_{c=1}^{\mathrm{C}}\); is built using Algorithm 6 on \(\mathbf{Y}^{\pi r}\) taking subspace dimension as equal to \(\min(20,p_{sr})\) (where \(p_{sr}\) is the dimension of source data samples), ratio \(\tau_{\text{max}}\) as equal to \(0.5\), and number of layers as equal to \(5\).