# An Information Theoretic Approach to Privacy-Preserving Interpretable and Transferable Learning
## 2 Mathematical Background
### Notations
1: Choose \(v\) and \(w=(w_{1},\cdots,w_{n})\) as in (12) and (14), respectively. 2: Choose a small positive value \(\kappa=10^{-1}\). 3: Set iteration count \(it=0\) and \(M_{|0}=M_{max}\). 4:while\(\tau(M|_{it},1)<\kappa\)do
5:\(M|_{it+1}=[0.9M|_{it}]\)
6:\(it\gets it+1\)
7:endwhile
8: Set \(M=M|_{it}\). 9:if\(\tau(M,1)\geq\frac{1}{p}\sum_{j=1}^{p}\mathrm{var}\Big{(}y^{1}_{j},\cdots,y^{ N}_{j}\Big{)}\)then
10:\(\sigma^{2}=1\)
11:else
12:\(\sigma^{2}=\frac{1}{\tau(M,1)}\frac{1}{p}\sum_{j=1}^{p}\mathrm{var}\Big{(}y^{ 1}_{j},\cdots,y^{N}_{j}\Big{)}\)
13:endif
14: Compute \(\mathsf{a}=\{a^{m}\}_{m=1}^{M}\) using (13), \(K_{\mathsf{xx}}\) using (9), \(K_{\mathsf{aa}}\) using (15), and \(K_{\mathsf{xa}}\) using (16). 15: Set \(\beta=1\). 16:repeat
17: Compute \(\alpha\) using (18). 18: Update the value of \(\beta\) using (19). 19:until (\(\beta\) nearly converges)
20: Compute \(\alpha\) using (18). 21:return the parameters set \(\mathbb{M}=\{\alpha,\mathsf{a},M,\sigma,w\}\). ```  
**Algorithm 1** Variational learning of the membership mappings [22]  
With reference to Algorithm 1, we have following:  
* The degrees of freedom associated to the Student-t membership mapping \(\
u\in\mathbb{R}_{+}\setminus[0,2]\) is chosen as \[\
u=2.1\] (12)
* The auxiliary inducing points are suggested to be chosen as the cluster centroids: \[\mathsf{a}=\{a^{m}\}_{m=1}^{M}=cluster\_centroid(\{x^{i}\}_{i=1}^{N},M)\] (13)where \(cluster\_centroid(\{x^{i}\}_{i=1}^{N},M)\) represents the k-means clustering on \(\{x^{i}\}_{i=1}^{N}\). * The parameters \((w_{1},\cdots,w_{n})\) for kernel function (10) are chosen such that \(w_{k}\) (for \(k\in\{1,2,\cdots,n\}\)) is given as \[\mathbb{F}(\mathcal{X}):=\{f:\mathcal{X}\to\mathbb{R}\mid f\text{ is }\mathcal{A}\text{-}\mathcal{B}(\mathbb{R})\text{ measurable}\}.\]0 (14) where \(x^{i}_{k}\) is the \(k\)-th element of vector \(x^{i}\in\mathbb{R}^{n}\). * \(K_{\text{aa}}\in\mathbb{R}^{M\times M}\) and \(K_{\text{xa}}\in\mathbb{R}^{N\times M}\) are matrices with their \((i,j)\)-th elements given as \[\mathbb{F}(\mathcal{X}):=\{f:\mathcal{X}\to\mathbb{R}\mid f\text{ is }\mathcal{A}\text{-}\mathcal{B}(\mathbb{R})\text{ measurable}\}.\]1 (15) \[\mathbb{F}(\mathcal{X}):=\{f:\mathcal{X}\to\mathbb{R}\mid f\text{ is }\mathcal{A}\text{-}\mathcal{B}(\mathbb{R})\text{ measurable}\}.\]2 (16) where \(kr:\mathbb{R}^{n}\times\mathbb{R}^{n}\rightarrow\mathbb{R}\) is a positive definite kernel function defined as in (10). * The scalar-valued function \(\tau(M,\sigma^{2})\) is defined as \[\mathbb{F}(\mathcal{X}):=\{f:\mathcal{X}\to\mathbb{R}\mid f\text{ is }\mathcal{A}\text{-}\mathcal{B}(\mathbb{R})\text{ measurable}\}.\]3 (17) where a is given by (13), \(\
u\) is given by (12), and parameters \((w_{1},\cdots,w_{n})\) (which are required to evaluate the kernel function for computing matrices \(K_{\text{xx}}\), \(K_{\text{aa}}\), and \(K_{\text{aa}}\)) are given by (14). * \(\alpha=[ \begin{smallmatrix}\alpha_{1}&\cdots&\alpha_{p}\end{smallmatrix} ]\in \mathbb{R}^{M\times p}\) is a matrix with its \(j\)-th column defined as \[\alpha_{j}:=\left(K_{\text{xa}}^{T}K_{\text{xa}}+\frac{Tr(K_{\text{xx}})-Tr(( K_{\text{aa}})^{-1}K_{\text{xa}}^{T}K_{\text{xa}})}{\
u+M-2}K_{\text{aa}}+\frac{K_{ \text{aa}}}{\beta}\right)^{-1}(K_{\text{xa}})^{T}\mathbf{y}_{j}\] (18)
* The disturbance precision value \(\beta\) is iteratively estimated as \[\frac{1}{\beta}=\frac{1}{pN}\sum_{j=1}^{p}\sum_{i=1}^{N}\left|y^{i}_{j}-\widehat {\mathcal{F}_{j}(x^{i})}\right|^{2}\] (19) where \(\widehat{\mathcal{F}_{j}(x^{i})}\) is the estimated membership-mapping output given as \[\widehat{\mathcal{F}_{j}(x^{i})}=\left(G(x^{i})\right)\alpha_{j}.\] (20) Here, \(G(x)\in\mathbb{R}^{1\times M}\) is a vector-valued function defined as \[G(x):=\left[ \begin{smallmatrix}kr(x,a^{1})&\cdots&kr(x,a^{M})\end{smallmatrix} \right]\] (21) where \(kr:\mathbb{R}^{n}\times\mathbb{R}^{n}\rightarrow\mathbb{R}\) is defined as in (10). **Definition 2** (Membership-Mappings Prediction [22]).