# An Information Theoretic Approach to Privacy-Preserving Interpretable and Transferable Learning
## 5 Information Theoretic Measures for Privacy Leakage, Interpretability, and Transferability
### A Unified Approach to Privacy-Preserving Interpretable and Transferable Learning
The first and second best performances have been marked. #### 6.3.2 Private Data

Here, we assume that heart rate values are private. As instantaneous heart rate is given as \(HR^{i}=60/RR^{i}\); thus, information about private data is directly contained in the R-R data vectors. #### 6.3.3 Semi-Supervised Transfer Learning Scenario

Out of the total subjects, a randomly chosen subject's data serve as the source domain data. Considering every other subject's data as the target domain data, the transfer learning experiment is performed independently on each target subject where 50% of the target subject's samples are labeled, and the remaining unlabeled target samples also serve as test data for evaluating the classification performance. However, only the target subjects, with data containing both the classes and at least 60 samples, were considered for experimentation. There are in total 48 such target subjects. #### 6.3.4 Experimental Design

Algorithm 9 is applied with \(d=1,\epsilon\in\{0.1,0.5,1,2,5,8,20,50,100,\infty\}\), and \(\delta=1\times 10^{-5}\). Each of the 48 experiments involves 10 different privacy-preserving semi-supervised transfer learning scenarios with privacy-loss bound values as \(\epsilon=0.1\), \(\epsilon=0.5\), \(\epsilon=1\), \(\epsilon=2\), \(\epsilon=5\), \(\epsilon=8\), \(\epsilon=20\), \(\epsilon=50\), \(\epsilon=100\), and \(\epsilon=\infty\). The following two requirements are associated to this application example:

1. The private source domain data must be protected while transferring knowledge from source to target domain; and
2. The interpretability of the source domain model should be high. In view of the aforementioned requirements, the models that correspond to the minimum privacy leakage and maximum interpretability measure amongst all the models obtained corresponding to 10 different choices of differential privacy-loss bound \(\epsilon\) are considered for detecting stress. #### 6.3.5 Results

Figure 6 summarizes the experimental results where accuracies obtained by both minimum privacy-leakage models and maximum interpretability-measure models have been displayed as box plots. It is observed in Figure 6 that the transfer and multi-task learning improves considerably the performance of source domain classifier. Table 17 reports the median values (of privacy leakage, interpretability measure, transferability measure, and classification accuracy) obtained in the experiments on 48 different subjects. The robust performance of transfer and multi-task learning scenario is further observed in Table 17.

Figure 6: The box plots of accuracies obtained in detecting mental stress on 48 different subjects. As a visualization example, Figure 7 displays the noise-added source domain heart rate interval data for different values of information theoretic measures. ## 7 Concluding Remarks

The paper has introduced information theoretic measures for privacy leakage, interpretability, and transferability to study the trade-offs. This is the first study to develop an information theory-based unified approach to privacy-preserving interpretable and transferable learning. The experiments have verified that the proposed measures (for privacy leakage, interpretability, and transferability) can be used to study the trade-off curves (between privacy leakage, interpretability measure, and transferability measure) and thus to optimize the models for the given application requirements such as the requirement of minimum privacy leakage, the requirement of maximum interpretability measure, and the requirement of maximum transferability measure. The experimental results on the MNIST dataset showed that the transfer and multi-task learning scenario improved remarkable the accuracy from 0.1760 to 0.9510 while ensuring the minimum privacy leakage.