# An Information Theoretic Approach to Privacy-Preserving Interpretable and Transferable Learning
## 5 Information Theoretic Measures for Privacy Leakage, Interpretability, and Transferability
### A Unified Approach to Privacy-Preserving Interpretable and Transferable Learning
* The results also verify the robust performance of Algorithm 9 under transfer and multi-task learning scenario, since the classification performance in the transfer and multi-task learning scenario, unlike the performance of the source domain classifier, is not adversely affected by a reduction in privacy leakage, interpretability measure, and transferability measure as observed in Figure 4a,e,f. Table 3 reports the results obtained by the models that correspond to minimum privacy leakage, maximum interpretability measure, and maximum transferability measure. The robustness of transfer and multi-task learning scenario is further highlighted in Table 3. To achieve the minimum value of privacy leakage, the accuracy of source domain classifier must be decreased to 0.1760; however, the transfer and multi-task learning scenario achieves the minimum privacy leakage value with the accuracy of 0.9510. As observed in Table 3, the maximum transferability-measure models also correspond to the maximum interpretability-measure models. Figure 4: The plots between privacy leakage, interpretability measure, transferability measure, and accuracy for MNIST dataset. ### Office and Caltech256 Datasets

The \"Office+Caltech256\" dataset has 10 common categories of both Office and Caltech256 datasets. The dataset has four domains: _amazon_, _webcam_, _dslr_, and _caltech256_. This dataset has been widely used [31; 32; 33; 34] for evaluating multi-class accuracy performance in a standard domain adaptation setting with a small number of labeled target samples. Following [32], the 4096-dimensional deep-net VGG-FC6 features are extracted from the images. However, for the learning of classifiers, the 4096-dimensional feature vectors are reduced to 100-dimensional feature vectors using principal components computed from the data of _amazon_ domain. Thus, corresponding to each image, a 100-dimensional data vector is constructed. #### 6.2.1 Interpretable Parameters

Corresponding to a data vector \(y\in\mathbb{R}^{100}\), an interpretable parameter vector \(t\in\{0,1\}^{10}\) is defined such that the \(j\)-th element \(t_{j}=1\), if the \(j\)-th class label is associated to \(y\); otherwise, \(t_{j}=0\). That is, interpretable vector \(t\), in our experimental setting, represents the class-label assigned to data vector \(y\). #### 6.2.2 Private Data

Here, we assume that the extracted image feature vectors are private, i.e., \(x_{sr}=y_{sr}\). #### 6.2.3 Semi-Supervised Transfer Learning Scenario

Similarly to [31; 32; 33; 34], the experimental setup is follows:

1. The number of training samples per class in the source domain is 20 for _amazon_ and is 8 for the other three domains;
2. The number of labeled samples per class in the target domain is 3 for all the four domains. Figure 5: An example of a source domain sample corresponding to different levels of privacy leakage, interpretability measure, and transferability measure. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **Privacy** & **Interpretability** & **Transferability** & **Classification** \\  & **Leakage** & **Measure** & **Measure** & **Accuracy** \\ \hline minimum privacy leakage & \(-\)50.72 & \(-\)2.14 & \(-\)664.52 & 0.9510 \\ transfer and multi-task learning & \(-\)50.72 & \(-\)2.14 & \(-\)664.52 & 0.1760 \\ minimum privacy leakage & \(-\)50.72 & \(-\)2.14 & \(-\)664.52 & 0.1760 \\ source domain classifier & & & & \\ \hline maximum interpretability measure & \(362.83\) & 5.44 & 451.93 & 0.9920 \\ transfer and multi-task learning & & & & \\ maximum interpretability measure & \(362.83\) & 5.44 & 451.93 & 0.9950 \\ source domain classifier & & & & \\ \hline maximum transferability measure & \(362.83\) & 5.44 & 451.93 & 0.9920 \\ transfer and multi-task learning & & & & \\ maximum transferability measure & \(362.83\) & 5.44 & 451.93 & 0.9950 \\ source domain classifier & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Results of experiments on MNIST dataset for evaluating privacy leakage, interpretability, and transferability.