# An Information Theoretic Approach to Privacy-Preserving Interpretable and Transferable Learning
## 5 Information Theoretic Measures for Privacy Leakage, Interpretability, and Transferability
### A Unified Approach to Privacy-Preserving Interpretable and Transferable Learning
3: Taking subspace dimension \(n_{\pi}=\min(\lceil p_{sr}/2\rceil,p_{rg})\), the source domain transformation matrix \(V^{+\pi r}\in\mathbb{R}^{n_{\pi}+\tau_{\text{Par}}}\) is defined as with its \(i\)-th row equal to the transpose of the eigenvector corresponding to the \(i\)-th largest eigenvalue of sample covariance matrix computed on differentially private approximated source samples. The target domain transformation matrix \(V^{\mathrm{fg}}\in\mathbb{R}^{n_{\pi}+\tau_{\text{Par}}}\) is defined as with its \(i\)-th row equal to the transpose of the eigenvector corresponding to the \(i\)-th largest eigenvalue of the sample covariance matrix computed on target samples. 4: For the case of heterogenous source and target domains, the subspace covariance matrix computed is used to transform target samples via (40) and (41) for defining the sets \(\{\mathbf{Y}_{c}^{\mathrm{g}+\pi}\}_{c=1}^{\mathrm{C}}\) and \(\mathbf{Y}_{*}^{\mathrm{g}+\pi r}\). 5: Initial target domain classifier, \(\{\mathcal{P}_{c}^{\mathrm{g}}|\}_{c=1}^{\mathrm{C}}\), is built using Algorithm 4 on labeled target samples, \(\{\mathbf{Y}_{c}^{\mathrm{g}+\pi}\}_{c=1}^{\mathrm{C}}\), taking subspace dimension as equal to \(\min(20,\min_{1\leq c\leq\{N_{c}^{\mathrm{g}}\}}-1)\) (where \(N_{c}^{\mathrm{g}}\) is the number of \(c\)-th class labeled target samples), ratio \(\tau_{\text{max}}\) as equal to \(1\), and number of layers as equal to \(1\). 6: The target domain classifier is updated using (42) and (43) until 4 iterations taking the monotonically non-decreasing subspace dimension \(n\) sequence as \(\{\min(5,p_{sr}),\min(10,p_{sr}),\min(15,p_{sr}),\min(20,p_{sr})\}\) and \(r_{\max-0.5}\). 7: The mapping from source to target domain is learned by means of a model, \(\mathbb{M}^{\pi r\to t_{\text{g}}}\), defined as in (44). 8: Compute privacy leakage, \(IL_{f_{\sum_{s=y_{sr}}^{+}}}\), and adversary model, \(\mathbb{BM}^{\mathbb{M}^{\pi r\to x_{sr}}}\), via applying Algorithm 8 on \(\{(y_{sr}^{+},x_{sr})\mid y_{sr}^{+}=f_{x_{sr}\to y_{sr}^{+}}(x_{sr}),\ x_{sr}\in \mathbf{X}^{\pi r},\)\(y_{sr}^{+}\in\mathbf{Y}^{+\pi r}\}\). 9: Compute interpretability measure, \(IL_{f_{\sum_{s=y_{sr}}^{+}}}\), and interpretability model, \(\mathbb{BM}^{\mathbb{M}^{\pi r\to t_{\text{g}}}}\), via applying Algorithm 8 on \(\{(y_{sr}^{+},t_{sr})\mid y_{sr}^{+}=f_{t_{sr}\to y_{sr}^{+}}(t_{sr}),\)\(t_{sr}\ \in\ \mathbf{T}^{\pi r},\ y_{sr}^{+}\in\mathbf{Y}^{+\pi r}\}\). 10: Compute transferability measure, \(IL_{f_{\sum_{s=y_{sr}}^{+}}}\), via applying Algorithm 8 on \(\{\left(y_{sr}^{+},t_{sr}\right)\mid y_{sr}^{+}=f_{t_{sr}\to y_{sr}^{+}}(t_{sr}),\)\(t_{sr}\ \in\ \mathbf{T}^{\pi r},\ y_{sr}^{+}\in\mathbf{Y}^{+\pi r}\}\). 11: Compute transferability measure, \(IL_{f_{\sum_{s=y_{sr}}^{+}}}\), via applying Algorithm 8 on \(\left\{\left(\hat{y}_{rg}^{\mathrm{fg}}(y_{rg}),\hat{y}_{rg}^{\mathrm{fg}}(y_{rg}) \right)\mid y_{rg}\in\{\mathbf{Y}_{c}^{\mathrm{g}}\}_{c=1}^{\mathrm{C}}\cup \mathbf{Y}_{*}^{\mathrm{g}}\right\}\), where
12: \[\hat{y}_{rg}^{\mathrm{gr}}(y_{rg}) =\widehat{\mathcal{WD}}\Big{(}y_{rg-sr}(y_{rg});\mathcal{P}_{f_{ \sum_{s=y_{sr}}^{+}}(y_{rg})}^{\mathcal{P}_{f_{\sum_{s=y_{sr}}^{+}}(y_{rg})} ^{\mathcal{P}_{f_{\sum_{s=y_{sr}}}^{+}}(y_{rg})}\Big{)}\] (116) \[\hat{y}_{rg}^{\mathrm{fg}}(y_{rg}) =\widehat{\mathcal{WD}}\Big{(}y_{rg-sr}(y_{rg});\mathcal{P}_{f_{ \sum_{s=y_{sr}}^{+}}(y_{rg})}^{\mathcal{P}_{f_{\sum_{s=y_{sr}}^{+}}(y_{rg})} ^{\mathcal{P}_{f_{\sum_{s=y_{sr}}^{+}}}(y_{rg})}\Big{)}\] (117) \[f_{y_{rg-sr}}(y_{rg}) =\hat{\mathcal{C}}\Big{(}y_{rg}-sr(y_{rg});\{\mathcal{P}_{c}^{ \mathrm{g}}\}_{c=1}^{\mathrm{C}},\{\mathcal{P}_{c}^{\mathrm{g}+\pi r}\}_{c=1}^{ \mathrm{C}},\mathbb{M}^{\pi r\to t_{\text{g}}}\Big{)},\] (118)

\(y_{lg\to t_{\text{g}}}(y_{rg})\) is defined as in (39), and \(\hat{\mathcal{C}}(\cdot)\) is defined by (47).