# Multiparty Data Publishing via Blockchain and Differential Privacy

Authors: Zhen Gu; Kejia Zhang; Guoyin Zhang
Journal: Security and Communication Networks
 (2022)

---

## Abstract

Data are distributed between different parties. Collecting data from multiple parties for analysis and mining will serve people better. However, it also brings unprecedented privacy threats to the participants. Therefore, safe and reliable data publishing among multiple data owners is an urgent problem to be solved. We mainly study the problem of privacy protection in data publishing. For a centralized scenario, we propose the LDA-DP algorithm. First, the within-class mean vectors and the pooled within-class scatter matrix are perturbed by the Gaussian noise. Second, the optimal projection direction vector with differential privacy is obtained by the Fisher criterion. Finally, the low-dimensional projection data of the original data are obtained. For distributed scenarios, we propose the Mul-LDA-DP algorithm based on a blockchain and differential privacy technology. First, the within-class mean vectors and within-class scatter matrices of local data are perturbed by the Gaussian noise and uploaded to the blockchain network. Second, the projection direction vector is calculated in the blockchain network and returned to the data owner. Finally, the data owner uses the projection direction vector to generate low-dimensional projection data of the original data and upload it to the blockchain network for publishing. Furthermore, in a distributed scenario, we propose a correlated noise generation scheme that uses the additivity of the Gaussian distribution to mitigate the effects of noise and can achieve the same noise level as the centralized scenario. We measure the utility of the published data by the SVM misclassification rate. We conduct comparative experiments with similar algorithms on different real data sets. The experimental results show that the data released by the two algorithms can maintain good utility in SVM classification.


---

## Body

## 1. Introduction

With the development of science and technology, effective data collection and analysis can help people make better decisions in production. For example, analyzing the information of the patient can help doctors improve the accuracy of diagnosis and level of medical services, and analyzing the trajectory data can improve city traffic congestion. The data contain sensitive information and need to be processed for privacy protection before publishing [1, 2]. There have been some studies on privacy preserving data publishing. For example, the k -anonymity privacy protection technology [3], the encryption technology [4, 5], the blockchain technology [6–8], and differential privacy technology [9–11]. Differential privacy has been widely used for privacy protection in recent years, the principle of differential privacy is to add random noise to data, which makes the attacker unable to distinguish the original input data. Differential privacy can quantitatively measure the degree of privacy protection and can resist attacks from attackers with background knowledge. Privacy preserving data publishing based on differential privacy has become a research hot spot [12–15].However, in the distributed scenario, data are possessed by multiple data owners. Data from a single data owner may not be sufficient for statistical learning, and aggregating data by a single data owner may not be possible. For example [16], in Table 1, the data are possessed by three data owners. Each row in Table 1 represents the information of an individual, where records 1 to 4 are from data owner 1, records 5 to 8 are from data owner 2, and records 9 to 10 are from data owner 3. Simply integrating and publishing the data from each data owner will cause a serious privacy leakage. Sharing and exchange of data in a distributed environment requires security guarantees. In order to solve the proposed problem, we make the following contributions:(1)
We propose two algorithms which are called LDA-DP and Mul-LDA-DP. The LDA-DP algorithm is used for privacy protection of data publishing in centralized scenario, and the Mul-LDA-DP algorithm is used for privacy protection of data publishing in distributed scenario.(2)
In the distributed scenario, the data owners cooperate with each other to publish a projection data set which satisfies differential privacy. In order to improve the utility of the published data in the distributed scenario, we propose a correlated noise generation scheme that uses the additivity of the Gaussian distribution to mitigate the effects of noise and can achieve the same noise level as the centralized scenario.(3)
We conduct experiments on different data sets. The experimental results show that the data released by LDA-DP and Mul-LDA-DP algorithms can maintain good utility in SVM classification.Table 1
Aggregated dataset of each data owner.
IDAgeJobGenderhours-per-weekincome139ShopkeeperMale40>50K255LawyerMale13≤50K338DancerMale20≤50K430DancerMale25≤50K528BuilderFemale40>50K637DancerFemale23≤50K749TeacherFemale16≤50K852BuilderMale45>50K931LawyerFemale50>50K1042BuilderMale40>50K


## 2. Related Work

In this section, we introduce the research status of privacy preserving data publishing in centralized scenario and distributed scenario, respectively.


### 2.1. Privacy Preserving Data Publishing in Centralized Scenario

Blum et al. [17] proposed the sublinear query (SULQ) input perturbation framework which adds noise to the covariance matrix, the framework can only be used for querying the projected subspace. Chaudhuri et al. [18] proposed the PPCA algorithm which is the improvement of SUQL algorithm. The PPCA algorithm randomly samples a k-dimensional subspace which ensures differential privacy and is biased toward high utility. Both SUQL and PPCA procedures are differentially private approximations to the top-k subspace. Zhang et al. [19] proposed the PrivBayes algorithm; first, they constructed a Bayesian network with differential privacy, and then they used the Bayesian network to generate a data set for publication. Chen et al. [20] presented the JTree algorithm. First, they explored the relationship between the attributes based on the sparse vector sampling technology, and then they constructed a Markov network that satisfies differential privacy and generated a synthetic data set for publication. Zhang et al. [21] proposed the PrivHD algorithm based on the JTree. They used high-pass filtering techniques to speed up the construction of Markov network and built a better joint tree for generating synthetic data set for publication. Xu et al. [22] proposed the DPPro algorithm; first, they randomly projected the original high-dimensional data into a low-dimensional space, and then they added noise to the projection vector and low-dimensional projection data; finally, they released the low-dimensional projection data. Zhang et al. [23] presented the PrivMN method. They constructed a Markov model with differential privacy, and then used the Markov model to generate a synthetic data set for publication. The algorithms mentioned above are mainly used for privacy preserving data publishing in centralized scenarios.


### 2.2. Privacy Preserving Data Publishing in Distributed Scenario

There are fewer researches on privacy protection of horizontally partitioned data publication. Ge et al. [24] proposed a distributed principal component analysis (DPS-PCA) algorithm with differential privacy; first, data owners collaborated to analyze the principal components, while protecting the private information, and then they released low-dimensional subspaces of high-dimensional sparse data. Wang et al. [25] proposed an efficient and scalable protocol for computing principal components in a distributed environment. First, the data owner encrypted the shared data and sent them to the semitrusted third party, then the semitrusted third party performed a private aggregation algorithm on the encrypted data and sent the aggregated data to data user for calculating the principal components. Imtiaz et al. [26] presented a distributed principal component analysis (DPdisPCA) algorithm with differential privacy. Each data owner used Gaussian noise to perturbed the local covariance matrix, and with the assistance of a semitrusted third party to calculate the principal components while ensuring local data privacy. Alhadidi et al. [27] proposed a two-party data publishing algorithm with differential privacy. They first presented a two-party protocol for the exponential mechanism which can be used as a subprotocol, the data released by this algorithm are suitable for classification tasks. Cheng et al. [28] proposed a differential privacy sequential update of the Bayesian network algorithm which is called DP-SUBN3, data owners collaboratively constructed the Bayesian network, data owners can treat the intermediate results as prior knowledge to construct the Bayesian network, and then they used the Bayesian network to generate a data set for publication. Wang et al. [29] proposed a distributed differential privacy anonymous algorithm and guaranteed that each step of the algorithm satisfies the definition of secure two-party computation. This is the first research about differentially private data publishing for arbitrarily partitioned data. In our prior work [16], we proposed the PPCA-DP-MH algorithm. First, data owners and a semitrusted third party cooperated to reduce the dimension of high-dimensional data to obtain the top k principal components that satisfy differential privacy, and then each data owner used the generative model of probabilistic principal component analysis to generate a data set with the same scale as the original data for publication. Different from the prior work [16], this paper uses the linear discriminant analysis to publish the projection data with differential privacy. Linear discriminant analysis can retain the class information of the data while reducing the dimension, which is beneficial to maintain the utility of the published data in classification.


## 2.1. Privacy Preserving Data Publishing in Centralized Scenario

Blum et al. [17] proposed the sublinear query (SULQ) input perturbation framework which adds noise to the covariance matrix, the framework can only be used for querying the projected subspace. Chaudhuri et al. [18] proposed the PPCA algorithm which is the improvement of SUQL algorithm. The PPCA algorithm randomly samples a k-dimensional subspace which ensures differential privacy and is biased toward high utility. Both SUQL and PPCA procedures are differentially private approximations to the top-k subspace. Zhang et al. [19] proposed the PrivBayes algorithm; first, they constructed a Bayesian network with differential privacy, and then they used the Bayesian network to generate a data set for publication. Chen et al. [20] presented the JTree algorithm. First, they explored the relationship between the attributes based on the sparse vector sampling technology, and then they constructed a Markov network that satisfies differential privacy and generated a synthetic data set for publication. Zhang et al. [21] proposed the PrivHD algorithm based on the JTree. They used high-pass filtering techniques to speed up the construction of Markov network and built a better joint tree for generating synthetic data set for publication. Xu et al. [22] proposed the DPPro algorithm; first, they randomly projected the original high-dimensional data into a low-dimensional space, and then they added noise to the projection vector and low-dimensional projection data; finally, they released the low-dimensional projection data. Zhang et al. [23] presented the PrivMN method. They constructed a Markov model with differential privacy, and then used the Markov model to generate a synthetic data set for publication. The algorithms mentioned above are mainly used for privacy preserving data publishing in centralized scenarios.


## 2.2. Privacy Preserving Data Publishing in Distributed Scenario

There are fewer researches on privacy protection of horizontally partitioned data publication. Ge et al. [24] proposed a distributed principal component analysis (DPS-PCA) algorithm with differential privacy; first, data owners collaborated to analyze the principal components, while protecting the private information, and then they released low-dimensional subspaces of high-dimensional sparse data. Wang et al. [25] proposed an efficient and scalable protocol for computing principal components in a distributed environment. First, the data owner encrypted the shared data and sent them to the semitrusted third party, then the semitrusted third party performed a private aggregation algorithm on the encrypted data and sent the aggregated data to data user for calculating the principal components. Imtiaz et al. [26] presented a distributed principal component analysis (DPdisPCA) algorithm with differential privacy. Each data owner used Gaussian noise to perturbed the local covariance matrix, and with the assistance of a semitrusted third party to calculate the principal components while ensuring local data privacy. Alhadidi et al. [27] proposed a two-party data publishing algorithm with differential privacy. They first presented a two-party protocol for the exponential mechanism which can be used as a subprotocol, the data released by this algorithm are suitable for classification tasks. Cheng et al. [28] proposed a differential privacy sequential update of the Bayesian network algorithm which is called DP-SUBN3, data owners collaboratively constructed the Bayesian network, data owners can treat the intermediate results as prior knowledge to construct the Bayesian network, and then they used the Bayesian network to generate a data set for publication. Wang et al. [29] proposed a distributed differential privacy anonymous algorithm and guaranteed that each step of the algorithm satisfies the definition of secure two-party computation. This is the first research about differentially private data publishing for arbitrarily partitioned data. In our prior work [16], we proposed the PPCA-DP-MH algorithm. First, data owners and a semitrusted third party cooperated to reduce the dimension of high-dimensional data to obtain the top k principal components that satisfy differential privacy, and then each data owner used the generative model of probabilistic principal component analysis to generate a data set with the same scale as the original data for publication. Different from the prior work [16], this paper uses the linear discriminant analysis to publish the projection data with differential privacy. Linear discriminant analysis can retain the class information of the data while reducing the dimension, which is beneficial to maintain the utility of the published data in classification.


## 3. Preliminaries

### 3.1. Linear Discriminant Analysis (LDA)

Linear discriminant analysis proposed by Fisher is one of the most widely used and extremely effective methods in the field of dimensionality reduction and pattern recognition. Its typical applications include face recognition, target tracking and detection, credit card fraud detection, and speech recognition. The idea of linear discriminant analysis for binary classification is to choose the projection direction so that the samples of different classes after projection are as far apart as possible and the samples within each class are as clustered as possible. We denote the data set asX=X1∪X2, Xk=x1k,x2k,…,xNkk,k=1,2. N=N1+N2. The within-class mean vector of samples in the original sample space is as follows:(1)μk=1Nk∑x∈Xkx,k=1,2.The between-class scatter matrix is as follows:(2)Sb=μ1−μ2μ1−μ2T.The within-class scatter matrix is as follows:(3)Sk=∑x∈XkxxT−NkμkμkT,k=1,2.Then, the pooled within-class scatter matrix is as follows:(4)Sw=S1+S2.It can also be expressed as follows:(5)Sw=∑k=12∑x∈XkxxT−∑k=12NkμkμkT.The criterion of Fisher is as follows:(6)maxwwTSb−1wwTSw−1w.Using the Lagrange multiplier method to find the optimal projection direction vector, we obtain the following:(7)w=Sw−1μ1−μ2.The result of linear discriminant analysis only gives the optimal projection direction, and does not give a clear classification result.


### 3.2. Differential Privacy

Differential privacy provides a rigorous privacy protection for sensitive information, it can be quantified by mathematical formulas. The essence of differential privacy is to use noise to randomly perturb the output results, so that it is difficult to distinguish the original input data according to the output results.Definition 1.
[30] A randomized algorithm ℳ is ε -indistinguishable if for any two neighboring databases D and D^ differing in a single entry, and for all O⊆Rangeℳ:(8)lnPrℳD∈OPrℳD^∈O≤ε,where ε is a small positive real number.
Whenε is small, ln1+ε≈ε, so PrℳD∈O/PrℳD^∈O∈1−ε,1+ε, ε is used to control the probability ratio of algorithm ℳ to obtain the same output on two neighboring databases, which reflects the level of privacy protection that ℳ can provide.Definition 2.
[30]. A randomized algorithm ℳ is ε,δ differential privacy, if for any two neighboring databases D and D^ differing in a single entry, and for any OO⊆Rangeℳ there is the following:(9)PrℳD∈O≤eεPrℳD^∈O+δ,where ε is a small positive real number called privacy budget and δ is a small positive real number. It is also called δ-approximate ε-indistinguishability.Definition 3.
is the relaxed version of differential privacy. Whenδ=0, it becomes Definition 1, which is the strict version of differential privacy. Formula (9) means that it is allowed to break the limit of formula (8) with a small probability δ.Theorem 1 ([31]).
The sufficient condition for the random functionℳ to satisfy ε,δ differential privacy is as follows:(10)PrlnPrℳD∈OPrℳD^∈O>ε≤δ,O⊆Rangeℳ.Theorem 2 (Sequential Composition) [31].
Letℳi be an εi,δi differentially private algorithm, i=1,2,…,n, then for the same data set D, the combined algorithm ℳℳ1D,ℳ2D,…,ℳnD is ∑i=1nεi,∑i=1nδi differential privacy.Theorem 3 (Parallel Composition) [31].
Letℳi be an εi,δi differentially private algorithm, i=1,2,…,n, D1,D2,…,Dn are disjoint data sets, the combined algorithm ℳℳ1D1,ℳ2D2,…,ℳnDn is max1≤i≤nεi,δi differential privacy.Theorem 4 (Post Processing) [31].
Letℳ:D⟶R be a randomized algorithm that is ε,δ differential privacy, let f:R⟶R′ be an arbitrary mapping, then f°ℳ:D⟶R′ is ε,δ differential privacy.


## 3.1. Linear Discriminant Analysis (LDA)

Linear discriminant analysis proposed by Fisher is one of the most widely used and extremely effective methods in the field of dimensionality reduction and pattern recognition. Its typical applications include face recognition, target tracking and detection, credit card fraud detection, and speech recognition. The idea of linear discriminant analysis for binary classification is to choose the projection direction so that the samples of different classes after projection are as far apart as possible and the samples within each class are as clustered as possible. We denote the data set asX=X1∪X2, Xk=x1k,x2k,…,xNkk,k=1,2. N=N1+N2. The within-class mean vector of samples in the original sample space is as follows:(1)μk=1Nk∑x∈Xkx,k=1,2.The between-class scatter matrix is as follows:(2)Sb=μ1−μ2μ1−μ2T.The within-class scatter matrix is as follows:(3)Sk=∑x∈XkxxT−NkμkμkT,k=1,2.Then, the pooled within-class scatter matrix is as follows:(4)Sw=S1+S2.It can also be expressed as follows:(5)Sw=∑k=12∑x∈XkxxT−∑k=12NkμkμkT.The criterion of Fisher is as follows:(6)maxwwTSb−1wwTSw−1w.Using the Lagrange multiplier method to find the optimal projection direction vector, we obtain the following:(7)w=Sw−1μ1−μ2.The result of linear discriminant analysis only gives the optimal projection direction, and does not give a clear classification result.


## 3.2. Differential Privacy

Differential privacy provides a rigorous privacy protection for sensitive information, it can be quantified by mathematical formulas. The essence of differential privacy is to use noise to randomly perturb the output results, so that it is difficult to distinguish the original input data according to the output results.Definition 1.
[30] A randomized algorithm ℳ is ε -indistinguishable if for any two neighboring databases D and D^ differing in a single entry, and for all O⊆Rangeℳ:(8)lnPrℳD∈OPrℳD^∈O≤ε,where ε is a small positive real number.
Whenε is small, ln1+ε≈ε, so PrℳD∈O/PrℳD^∈O∈1−ε,1+ε, ε is used to control the probability ratio of algorithm ℳ to obtain the same output on two neighboring databases, which reflects the level of privacy protection that ℳ can provide.Definition 2.
[30]. A randomized algorithm ℳ is ε,δ differential privacy, if for any two neighboring databases D and D^ differing in a single entry, and for any OO⊆Rangeℳ there is the following:(9)PrℳD∈O≤eεPrℳD^∈O+δ,where ε is a small positive real number called privacy budget and δ is a small positive real number. It is also called δ-approximate ε-indistinguishability.Definition 3.
is the relaxed version of differential privacy. Whenδ=0, it becomes Definition 1, which is the strict version of differential privacy. Formula (9) means that it is allowed to break the limit of formula (8) with a small probability δ.Theorem 1 ([31]).
The sufficient condition for the random functionℳ to satisfy ε,δ differential privacy is as follows:(10)PrlnPrℳD∈OPrℳD^∈O>ε≤δ,O⊆Rangeℳ.Theorem 2 (Sequential Composition) [31].
Letℳi be an εi,δi differentially private algorithm, i=1,2,…,n, then for the same data set D, the combined algorithm ℳℳ1D,ℳ2D,…,ℳnD is ∑i=1nεi,∑i=1nδi differential privacy.Theorem 3 (Parallel Composition) [31].
Letℳi be an εi,δi differentially private algorithm, i=1,2,…,n, D1,D2,…,Dn are disjoint data sets, the combined algorithm ℳℳ1D1,ℳ2D2,…,ℳnDn is max1≤i≤nεi,δi differential privacy.Theorem 4 (Post Processing) [31].
Letℳ:D⟶R be a randomized algorithm that is ε,δ differential privacy, let f:R⟶R′ be an arbitrary mapping, then f°ℳ:D⟶R′ is ε,δ differential privacy.


## 4. Proposed Methods

In this section, we will propose two algorithms which are called LDA-DP and Mul-LDA-DP. The LDA-DP algorithm is used for privacy protection of data publishing in the centralized scenario, and the Mul-LDA-DP algorithm is used for privacy protection of data publishing in the distributed scenario. Without loss of generality, we assume that all individual data in this paper are normalized top -dimensional unit vectors.


### 4.1. LDA-DP Algorithm

In this section, we propose the LDA-DP algorithm for centralized data publishing.


#### 4.1.1. Problem Statement and Algorithm Proposed

The data setX contains two classes of data individuals denoted as X=X1∪X2, where Xk=x1k,x2k,…,xNkk,k=1,2. Our goal is to protect the privacy information of the original data from being leaked while publishing the projection data of the original data.In order to solve this problem, we propose the LDA-DP algorithm, which is mainly divided into two stages. First, we use the Gaussian mechanism of differential privacy to perturb the within-class mean vectorsμkk=1,2. Second, we use the Gaussian mechanism to perturb the pooled within-class scatter matrix Sw. Finally, we get the projection direction vector w that satisfies ε,δ differential privacy and publish the low-dimensional projected data of the original data. The specific details are in Algorithm 1.Algorithm 1:LDA-DP algorithm.
Input: Data sets X, privacy parameters ε1,δ1, ε2,δ2Output: Projection direction vector w, projection data X˜(1)
fork=1 to 2 do(2)
Setσ1=p3/2ln2/πδ12+p3ln2/πδ12+2ε1/ε1, which generates a p dimension noise vector gk; each entry is sampled from N0,σ12(3)
Computesμk=1/Nk∑x∈Xkx+gk(4)
end for(5)
returnμk,k=1,2(6)
Setσ2=p+1ln2/πδ22+p+12ln2/πδ22+4ε2/2ε2, which generates a p×p random matrix G. Let G be a symmetric matrix with the upper triangle (including the diagonal) entries are sampled from N0,σ22 and make the symmetrical position entries in the lower triangle matrix equal to the upper triangle.(7)
ComputesSw=∑x∈Xk2xxT−∑k=12NkμkμkT+G(8)
Computesw=Sw−1μ1−μ2(9)
ComputesX˜=Xw


#### 4.1.2. Privacy Analysis of LDA-DP Algorithm

Theorem 5.
The within-class mean vectorμkk=1,2 in Algorithm 1 satisfies ε1,δ1 differential privacy when each entry of gkk=1,2 is sampled from N0,σ12, where σ1≥p3/2ln2/πδ12+p3ln2/πδ12+2ε1/ε1,0<δ1<2/π.Proof. We denote the two neighboring data sets areX=X1∪X2 and X^=X^1∪X^2, where only one individual is different, without losing general assumption. Suppose the different individuals are in X1 and X^1, we denote them as xN1≠x^N1, they are p -dimensional unit vector. We denote a=∑x∈X1x and a^=∑x∈X1x, let c=a+g1 and c^=a^+g^1, each entry of g1 and g^1 is sampled from N0,σ12.The log ratio of the probabilitiesc and c^ at a point h is lnPc=h|X/Pc^=h|X^, the numerator in the ratio describes the probability of seeing h when the data set is X, the denominator corresponds the probability of seeing this same value when the data set is X^.By Theorem1, we will to find the value of σ1 such that the inequality lnPc=h|X/Pc^=h|X^=lnPh−a|X/Ph−a^|X^≤ε1 holds at least with probability 1−δ1.(11)lnPh−a|XPh−a^|X^=12σ12∑i=1phi−a^i2−hi−ai2=12σ12∑i=1p2hi−aiai−a^i+ai−a^i2≤12σ12∑i=1p2hi−aixN1i−x^N1i+12σ12∑i=1pxN1i−x^N1i2.Using the Lagrange multiplier method, we can get the maximum value of the objective function∑i=1pxN1i+x^N1i is 2p under the condition of ∑i=1pxN1i2=1,∑i=1px^N1i2=1.Then, we can obtain:∑i=1pxN1i−x^N1i≤∑i=1pxN1i+x^N1i≤2p. Similarly, we can obtain the following:(12)∑i=1pxN1i−x^N1i2≤4.So,1/2σ12∑i=1p2hi−aixN1i−x^N1i≤2p3/2r/σ12, where gi1=hi−ai≤r, for all i, and 1/2σ12∑i=1pxN1i−x^N1i2≤2/σ12.Then,lnPb=h|X/Pb^=h|X^≤2p3/2r+2/σ12, this quantity is bounded by ε1 whenever gi1≤r≤ε1σ12−2/2p3/2.To ensure privacy loss bounded byε1 with probability at least 1−δ1, we require to find σ1 that satisfies this inequality Prgi1≥ε1σ12−2/2p3/2≤δ1, due to symmetry, we will find σ1 such that Prgi1≥ε1σ12−2/2p3/2≤δ1/2.The tail bound is as follows:(13)Prgi1>t=∫t+∞12πσ1e−x22σ12dx=x=t+y∫0+∞12πσ1e−t+y22σ12dy,≤12πσ1e−t22σ12∫0+∞e−tyσ12dy≤1tσ12πe−t22σ12.We lett=ε1σ12−2/2p3/2, then 1/tσ1/2πe−t2/2σ12≤δ1/2, then we obtain the following:(14)lntσ1+t22σ12≥ln22πδ1.Whenσ1≥2p3/2+4p3+8ε1/2ε1, the first term in (14) is non-negative. To make the inequality (14) hold, we let t2/2σ12=1/2σ12ε1σ12−2/2p3/22≥ln2/2πδ1, then we obtain the following:(15)σ1≥p3/2ln2/πδ12+p3ln2/πδ12+2ε1ε1,0<δ1<2π.Theorem 6.
The pooled within-class scatter matrixSw in Algorithm 1 satisfies ε2,δ2 differential privacy, when each entry in the symmetric random matrix G is sampled from N0,σ22, where (16)σ2≥p+1ln2πδ22+p+12ln2πδ22+4ε2/2ε2,0<δ2<2π.Proof. Two neighboring data sets areX=X1∪X2 and X^=X^1∪X^2, where only one entry is different, without losing general assumption, suppose the different entry are in X1 and X^1 and denoted them as xN1≠x^N1.Becauseμkk=1,2 in (5) satisfies differential privacy has been proved by Theorem 5 which can be treated as a constant in (5), so if we want to prove that this theorem holds, it is only necessary to prove that the first item ∑k=12∑x∈XkxxT in (5) satisfies ε2,δ2 differential privacy after adding random matrix G.We denoteB=∑k=12∑x∈XkxxT, B^=∑k=12∑x∈XkxxT, let C=B+G and C^=B^+G^, G and G^ are two independent symmetric random matrices with the upper triangle (including the diagonal) entries are sampled from N0,σ22, and make the symmetrical position entries in the lower triangle matrix equal to the upper triangle.The log ratio of the probabilitiesC and C^ at a point H is lnPC=H|X/PC^=H|X^.By Theorem1, we need to find the value of σ2 such that the inequality lnPC=H|X/PC^=H|X^≤ε2 holds at least with probability 1−δ2.(17)lnPC=HXPC^=HX^=lnPH−BXPH−B^X^=12σ22∑1≤i≤j≤pHij−B^ij2−Hij−Bij2=12σ22∑1≤i≤j≤p2Hij−BijBij−B^ij+Bij−B^ij2≤12σ22∑1≤i≤j≤p2Hij−BijxN1ixN1j−x^N1ix^N1j+12σ22∑1≤i≤j≤pxN1ixN1j−x^N1ix^N1j2.By using the Lagrange multiplier method and the inequality in [18], the following inequalities hold:(18)∑1≤i≤j≤pxN1ixN1j−x^N1ix^N1j2≤2,∑1≤i≤j≤pxN1ixN1j−x^N1ix^N1j≤p+1.Then,lnPH−BX/PH−B^|X^|≤rp+1+1/σ22, where Gij=Hij−Bij≤r for all i,j.The rest of the proof process is similar to Theorem5, then we can obtain the following:(19)σ2≥p+1ln2/πδ22+p+12ln2/πδ22+4ε22ε2,0<δ2<2π.We have proven that the within-class mean vectorμkk=1,2 satisfies ε1,δ1 differential privacy, the pooled within-class scatter matrix Sw satisfies ε2,δ2 differential privacy, by the property of differential privacy sequential composition, the projection direction vector in the Algorithm 1 satisfies ε,δ differential privacy, where ε=ε1+ε2,δ=δ1+δ2. For the published projection data X˜=Xw, X∈RN×p, w∈Rp×1,p<N, we can regard X˜=Xw as a set of undetermined system of equation, the number of variables are more than equations, so the equation has infinitely many sets of solutions, that is, it is impossible to infer the information of the original data X from the published projection data X˜.


### 4.2. Mul-LDA-DP Algorithm

In this section, we propose the Mul-LDA-DP algorithm for distributed data publishing. The mathematical notations used in this section are summarized in Table2.Table 2
Summary of notations.
NotationExplanationMThe number of data ownersPmThem-th data ownerNmkThe number of individuals in thek-th class owned by PmNkThe total number of individuals in thek -th class, Nk=∑m=1MNmkXmkThe set of thek -th class data owned by PmXkThe set of thek -th class data. Xk=∪m=1MXmkXmThe data set owned byPm. Xm=∪k=12XmkμmkThe within-class mean vector of thek -th class data owned by PmμkThe within-class mean vector of thek -th class dataSmkThe within-class scatter matrix of thek -th class data owned by PmSkThe within-class scatter matrix of thek-th class dataSwThe pooled within-class scatter matrix


#### 4.2.1. Problem Statement and Algorithm Proposed

In the distributed scenario, data are stored by multiple data owners rather than a single owner, and the data owners do not trust each other. Data at a single site may not be sufficient for statistical learning. One solution is that each data owner uses the LDA-DP algorithm in Section4.1 to publish the projection data independently. Another solution is the data owners cooperate with each other to publish the projection data of the integrated data. Comparing the two solutions, it is obvious that the latter solution can improve the utility of publishing data. Based on the idea of the second solution and [32], we propose the Mul-LDA-DP algorithm for distributed data publishing. The entity description of the model is as follows.(1)
Data owner. The data ownerPmm=1,2,…,M has a data set Xm. Each data owner can generate random vectors and matrices to perturb the within-class mean vectors and within-class scatter matrices locally.(2)
Data publisher. The data publisher is a data publishing platform based on blockchain. The data publisher aggregates the local within-class mean vectors and within-class scatter matrices with noise. The data publisher can obtain the projection vector that satisfies differential privacy and publishes the projection data of the pooled data.(3)
A random number generator. It can generate random vectors and random matrices and send them to data owners and data publisher secretly.Threat Model. In our setting, we assume that the data owners and data publisher are honest-but-curious, that is, they follow the protocol but may try to deduce information of other data owners from the received messages.Two types of adversaries are considered, which are external attackers and internal attackers. External attackers which can be called an external eavesdropper may gain access to information such as data sent by data owners to the data publisher. Internal adversaries can be the data owners and the data publisher. The goal of each data owner is to extract the information not owned by him, while the goal of the data publisher is to extract the information from each data owner.Distributed Within-Class Mean Vectors and Pooled Within-Class Scatter Matrix Computation. When the data are owned byM data owners, the within-class mean vectors (1) can be decomposed into the following:(20)μk=1Nk∑m=1MNmkμmk,k=1,2,where μmk=1/Nmk∑x∈Xmkx.The pooled within-class scatter matrix (5) can be decomposed into the following:(21)Sw=∑k=12Sk=∑k=12∑m=1MSmk=∑m=1M∑k=12Smk,where Smk=∑x∈XmkxxT−NmkμmkμmkT.The abovementioned result allows each data owner to compute and perturb a partial result simultaneously locally. Therefore, we use the additivity of Gaussian distribution to propose a correlated noise generation scheme. We design the noise generation procedure such that (i) we can ensure that the data output from each data owner satisfy differential privacy and (ii) we can achieve the noise level of the same as the pooled data scenario.Scheme for Perturbing Shared Data by Correlated Noise. To prevent the data publisher and other data owners learning the privacy of local data, the data owner uses the noise generated by himself and the noise generated by the random number generator to perturb the local within-class mean vectors and within-class scatter matrices. Through our correlated noise design scheme, the data aggregated by the data publisher contain the same level of noise as the centralized scenario. The scheme is described as below:(1)
Initialization stage. The random number generator generatesp dimensional random vectors g¯mk, each entry is sampled from N0,M−1/Mσ12, generates p×p random matrices G¯m, let G¯m be the symmetric matrix with the upper triangle (including the diagonal) entries are sampled from N0,M−1/Mσ22, and makes the symmetrical position entries in the lower triangle matrix equal to the upper triangle, m=0,1,2,…,M,k=1,2. Make these random vectors and matrices satisfy ∑m=0Mg¯mk=0,∑m=0MG¯m=0p×p, then g¯mkk=1,2 and G¯m are sent to data owner Pm secretly, g¯0kk=1,2 and G¯0 are sent to the data publisher secretly.(2)
Data ownerPm generates p dimensional random vectors gmkk=1,2, each entry is sampled from N0,1/Mσ12, computes μmk=1/Nmk∑x∈Xmkx+gmk+g¯mk,k=1,2, and sends them to the data publisher.(3)
The data publisher computesμk=1/Nk∑m=1MNmkμmk+g¯0k,k=1,2 and sends them to each data owner.(4)
The data ownerPm generates p×p random matrix Gm, let Gm be the symmetric matrix with the upper triangle (including the diagonal) entries are sampled from N0,1/Mσ22, and make the symmetrical position entries in the lower triangle matrix equal to the upper triangle. Data owner Pm computes Sm=∑k=12Smk+Gm+G¯m and sends it to the data publisher.(5)
The data publisher computesSw=∑m=1MSm+G¯0 and calculates the projection vector w that satisfies differential privacy.The specific details of Mul-LDA-DP algorithm are in Algorithm 2. The input random vectorsg¯mk and random matrices G¯mk in Algorithm 2 are generated in the initialization stage by the random number generator, m=0,1,2,…,M,k=1,2.Algorithm 2:Mul-LDA-DP algorithm.
Input: Data sets Xm, m=1,2,…,M,k=1,2, privacy parameters ε1,δ1, ε2,δ2, random vector g¯mk and random matrix G¯m which are generated in initialization stage, m=0,1,2,…,M;k=1,2.Output: Projection direction vector w, projection data X˜(1)
form=1 to Mdo(2)
fork=1 to 2 do(3)
Setσ1=p3/2ln2/πδ12+p3ln2/πδ12+2ε1/ε1, data owner generates p dimensional random vector gmk, each entry is sampled from N0,σ12/M(4)
Computeμmk=1/Nmk∑x∈Xmkx+gmk+g¯mk(5)
end for(6)
end for(7)
Computeμk=1/Nk∑m=1MNmkμmk+g¯0k(8)
form=1 to M do(9)
Setσ2=p+1ln2/πδ22+p+12ln2/πδ22+4ε2/2ε2, data owner generates p×p symmetric random matrices Gm, each entry is sampled from N0,σ22/M(10)
fork=1 to 2 do(11)
ComputeSmk=∑x∈XmkxxT−NmkμmkμmkT(12)
end for(13)
ComputeSm=∑k=12Smk+Gm+G¯m(14)
end for(15)
ComputeSw=∑m=1MSm+G¯0(16)
Computew=Sw−1μ1−μ2(17)
returnX˜=∪m=1MXmw


#### 4.2.2. Privacy Analysis of the Mul-LDA-DP Algorithm

Theorem 7.
The within-class mean vectorμkk=1,2 in Algorithm 2 satisfies ε1,δ1 differential privacy.Proof.μmk=1/Nmk∑x∈Xmkx+gmk+g¯mk because each entry of gmk is sampled from N0,1/Mσ12, and each entry of g¯mk is sampled from N0,M−1/Mσ12, so each entry of gmk+g¯mk obeys N0,σ12. By Theorem 5, μmk satisfies ε1,δ1 differential privacy.Due to the post-processing property of differential privacy, the within-class mean vectorμk=1/Nk∑m=1MNmkμmk+g¯0k in Algorithm 2 satisfies ε1,δ1 differential privacy.Theorem 8.
The pooled within-class scatter matrixSw in Algorithm 2 satisfies ε2,δ2 differential privacy.Proof.Sm=∑k=12Smk+Gm+G¯m , where each entry of symmetric random matrix Gm is sampled from N0,1/Mσ22, and each entry of symmetric random matrix G¯m is sampled from N0,M−1/Mσ22, so each entry of G¯m+G¯m obeys N0,σ22. By Theorem 6, Sm satisfies ε2,δ2 differential privacy. Due to the post-processing property of differential privacy, the pooled within-class scatter matrix Sw=∑m=1MSm+G¯0 in Algorithm 2 satisfies ε2,δ2 differential privacy.We have proven bothμkk=1,2 and Sw satisfy differential privacy, we will show that the level of noise is the same as the centralized scenario. In the initialization stage, the noise vectors and matrices generated by the random number generator satisfy ∑m=0Mg¯mk=0 and ∑m=0MG¯m=0p×p.The within-class mean vectorμkk=1,2 is as follows:(22)μk=1Nk∑m=1MNmkμmk+g¯0k=1Nk∑m=1M∑x∈Xmkx+gmk+g¯mk+g¯0k,=1Nk∑m=1M∑x∈Xmkx+∑m=1Mgmk+∑m=0Mg¯mk=1Nk∑m=1M∑x∈Xmkx+∑m=1Mgmk,=1Nk∑x∈Xkx+∑m=1Mgmk.Each entry of∑m=1Mgmk obeys N0,σ12.The pooled within-class scatter matrixSw is as follows:(23)Sw=∑m=1MSm+G¯0=∑m=1M∑k=12Smk+Gm+G¯m+G¯0,=∑m=1M∑k=12Smk+∑m=1MGm+∑m=0MG¯m=∑k=12∑m=1MSmk+∑m=1MGm,=∑k=12Sk+∑m=1MGm.Each entry of∑m=1MGm obeys N0,σ22.According to Theorems5 and 6, the within-class mean vector μkk=1,2 and pooled within-class scatter matrix Sw contain the same level of noise as the centralized scenario, and we achieve the purpose of improving the utility of publishing data while protecting the data privacy.There are three opportunities for attackers to steal the data transmitted between the data owner and the data publisher. The first time is that the data owner sends the within-class mean vectors to the data publisher, the second time is that the data owner sends the within-class scatter matrices to data publisher. From Theorems7 and 8, we know that the within-class mean vectors and the within-class scatter matrices satisfy differential privacy. Therefore, the attacker cannot infer the information of the original data from the eavesdropped data. The third time is that the data owner sends projection data to the data publisher, in Section 4.1.2, we have analyzed that it is impossible to infer the information of the original data from the published projection data.


## 4.1. LDA-DP Algorithm

In this section, we propose the LDA-DP algorithm for centralized data publishing.


### 4.1.1. Problem Statement and Algorithm Proposed

The data setX contains two classes of data individuals denoted as X=X1∪X2, where Xk=x1k,x2k,…,xNkk,k=1,2. Our goal is to protect the privacy information of the original data from being leaked while publishing the projection data of the original data.In order to solve this problem, we propose the LDA-DP algorithm, which is mainly divided into two stages. First, we use the Gaussian mechanism of differential privacy to perturb the within-class mean vectorsμkk=1,2. Second, we use the Gaussian mechanism to perturb the pooled within-class scatter matrix Sw. Finally, we get the projection direction vector w that satisfies ε,δ differential privacy and publish the low-dimensional projected data of the original data. The specific details are in Algorithm 1.Algorithm 1:LDA-DP algorithm.
Input: Data sets X, privacy parameters ε1,δ1, ε2,δ2Output: Projection direction vector w, projection data X˜(1)
fork=1 to 2 do(2)
Setσ1=p3/2ln2/πδ12+p3ln2/πδ12+2ε1/ε1, which generates a p dimension noise vector gk; each entry is sampled from N0,σ12(3)
Computesμk=1/Nk∑x∈Xkx+gk(4)
end for(5)
returnμk,k=1,2(6)
Setσ2=p+1ln2/πδ22+p+12ln2/πδ22+4ε2/2ε2, which generates a p×p random matrix G. Let G be a symmetric matrix with the upper triangle (including the diagonal) entries are sampled from N0,σ22 and make the symmetrical position entries in the lower triangle matrix equal to the upper triangle.(7)
ComputesSw=∑x∈Xk2xxT−∑k=12NkμkμkT+G(8)
Computesw=Sw−1μ1−μ2(9)
ComputesX˜=Xw


### 4.1.2. Privacy Analysis of LDA-DP Algorithm

Theorem 5.
The within-class mean vectorμkk=1,2 in Algorithm 1 satisfies ε1,δ1 differential privacy when each entry of gkk=1,2 is sampled from N0,σ12, where σ1≥p3/2ln2/πδ12+p3ln2/πδ12+2ε1/ε1,0<δ1<2/π.Proof. We denote the two neighboring data sets areX=X1∪X2 and X^=X^1∪X^2, where only one individual is different, without losing general assumption. Suppose the different individuals are in X1 and X^1, we denote them as xN1≠x^N1, they are p -dimensional unit vector. We denote a=∑x∈X1x and a^=∑x∈X1x, let c=a+g1 and c^=a^+g^1, each entry of g1 and g^1 is sampled from N0,σ12.The log ratio of the probabilitiesc and c^ at a point h is lnPc=h|X/Pc^=h|X^, the numerator in the ratio describes the probability of seeing h when the data set is X, the denominator corresponds the probability of seeing this same value when the data set is X^.By Theorem1, we will to find the value of σ1 such that the inequality lnPc=h|X/Pc^=h|X^=lnPh−a|X/Ph−a^|X^≤ε1 holds at least with probability 1−δ1.(11)lnPh−a|XPh−a^|X^=12σ12∑i=1phi−a^i2−hi−ai2=12σ12∑i=1p2hi−aiai−a^i+ai−a^i2≤12σ12∑i=1p2hi−aixN1i−x^N1i+12σ12∑i=1pxN1i−x^N1i2.Using the Lagrange multiplier method, we can get the maximum value of the objective function∑i=1pxN1i+x^N1i is 2p under the condition of ∑i=1pxN1i2=1,∑i=1px^N1i2=1.Then, we can obtain:∑i=1pxN1i−x^N1i≤∑i=1pxN1i+x^N1i≤2p. Similarly, we can obtain the following:(12)∑i=1pxN1i−x^N1i2≤4.So,1/2σ12∑i=1p2hi−aixN1i−x^N1i≤2p3/2r/σ12, where gi1=hi−ai≤r, for all i, and 1/2σ12∑i=1pxN1i−x^N1i2≤2/σ12.Then,lnPb=h|X/Pb^=h|X^≤2p3/2r+2/σ12, this quantity is bounded by ε1 whenever gi1≤r≤ε1σ12−2/2p3/2.To ensure privacy loss bounded byε1 with probability at least 1−δ1, we require to find σ1 that satisfies this inequality Prgi1≥ε1σ12−2/2p3/2≤δ1, due to symmetry, we will find σ1 such that Prgi1≥ε1σ12−2/2p3/2≤δ1/2.The tail bound is as follows:(13)Prgi1>t=∫t+∞12πσ1e−x22σ12dx=x=t+y∫0+∞12πσ1e−t+y22σ12dy,≤12πσ1e−t22σ12∫0+∞e−tyσ12dy≤1tσ12πe−t22σ12.We lett=ε1σ12−2/2p3/2, then 1/tσ1/2πe−t2/2σ12≤δ1/2, then we obtain the following:(14)lntσ1+t22σ12≥ln22πδ1.Whenσ1≥2p3/2+4p3+8ε1/2ε1, the first term in (14) is non-negative. To make the inequality (14) hold, we let t2/2σ12=1/2σ12ε1σ12−2/2p3/22≥ln2/2πδ1, then we obtain the following:(15)σ1≥p3/2ln2/πδ12+p3ln2/πδ12+2ε1ε1,0<δ1<2π.Theorem 6.
The pooled within-class scatter matrixSw in Algorithm 1 satisfies ε2,δ2 differential privacy, when each entry in the symmetric random matrix G is sampled from N0,σ22, where (16)σ2≥p+1ln2πδ22+p+12ln2πδ22+4ε2/2ε2,0<δ2<2π.Proof. Two neighboring data sets areX=X1∪X2 and X^=X^1∪X^2, where only one entry is different, without losing general assumption, suppose the different entry are in X1 and X^1 and denoted them as xN1≠x^N1.Becauseμkk=1,2 in (5) satisfies differential privacy has been proved by Theorem 5 which can be treated as a constant in (5), so if we want to prove that this theorem holds, it is only necessary to prove that the first item ∑k=12∑x∈XkxxT in (5) satisfies ε2,δ2 differential privacy after adding random matrix G.We denoteB=∑k=12∑x∈XkxxT, B^=∑k=12∑x∈XkxxT, let C=B+G and C^=B^+G^, G and G^ are two independent symmetric random matrices with the upper triangle (including the diagonal) entries are sampled from N0,σ22, and make the symmetrical position entries in the lower triangle matrix equal to the upper triangle.The log ratio of the probabilitiesC and C^ at a point H is lnPC=H|X/PC^=H|X^.By Theorem1, we need to find the value of σ2 such that the inequality lnPC=H|X/PC^=H|X^≤ε2 holds at least with probability 1−δ2.(17)lnPC=HXPC^=HX^=lnPH−BXPH−B^X^=12σ22∑1≤i≤j≤pHij−B^ij2−Hij−Bij2=12σ22∑1≤i≤j≤p2Hij−BijBij−B^ij+Bij−B^ij2≤12σ22∑1≤i≤j≤p2Hij−BijxN1ixN1j−x^N1ix^N1j+12σ22∑1≤i≤j≤pxN1ixN1j−x^N1ix^N1j2.By using the Lagrange multiplier method and the inequality in [18], the following inequalities hold:(18)∑1≤i≤j≤pxN1ixN1j−x^N1ix^N1j2≤2,∑1≤i≤j≤pxN1ixN1j−x^N1ix^N1j≤p+1.Then,lnPH−BX/PH−B^|X^|≤rp+1+1/σ22, where Gij=Hij−Bij≤r for all i,j.The rest of the proof process is similar to Theorem5, then we can obtain the following:(19)σ2≥p+1ln2/πδ22+p+12ln2/πδ22+4ε22ε2,0<δ2<2π.We have proven that the within-class mean vectorμkk=1,2 satisfies ε1,δ1 differential privacy, the pooled within-class scatter matrix Sw satisfies ε2,δ2 differential privacy, by the property of differential privacy sequential composition, the projection direction vector in the Algorithm 1 satisfies ε,δ differential privacy, where ε=ε1+ε2,δ=δ1+δ2. For the published projection data X˜=Xw, X∈RN×p, w∈Rp×1,p<N, we can regard X˜=Xw as a set of undetermined system of equation, the number of variables are more than equations, so the equation has infinitely many sets of solutions, that is, it is impossible to infer the information of the original data X from the published projection data X˜.


## 4.1.1. Problem Statement and Algorithm Proposed

The data setX contains two classes of data individuals denoted as X=X1∪X2, where Xk=x1k,x2k,…,xNkk,k=1,2. Our goal is to protect the privacy information of the original data from being leaked while publishing the projection data of the original data.In order to solve this problem, we propose the LDA-DP algorithm, which is mainly divided into two stages. First, we use the Gaussian mechanism of differential privacy to perturb the within-class mean vectorsμkk=1,2. Second, we use the Gaussian mechanism to perturb the pooled within-class scatter matrix Sw. Finally, we get the projection direction vector w that satisfies ε,δ differential privacy and publish the low-dimensional projected data of the original data. The specific details are in Algorithm 1.Algorithm 1:LDA-DP algorithm.
Input: Data sets X, privacy parameters ε1,δ1, ε2,δ2Output: Projection direction vector w, projection data X˜(1)
fork=1 to 2 do(2)
Setσ1=p3/2ln2/πδ12+p3ln2/πδ12+2ε1/ε1, which generates a p dimension noise vector gk; each entry is sampled from N0,σ12(3)
Computesμk=1/Nk∑x∈Xkx+gk(4)
end for(5)
returnμk,k=1,2(6)
Setσ2=p+1ln2/πδ22+p+12ln2/πδ22+4ε2/2ε2, which generates a p×p random matrix G. Let G be a symmetric matrix with the upper triangle (including the diagonal) entries are sampled from N0,σ22 and make the symmetrical position entries in the lower triangle matrix equal to the upper triangle.(7)
ComputesSw=∑x∈Xk2xxT−∑k=12NkμkμkT+G(8)
Computesw=Sw−1μ1−μ2(9)
ComputesX˜=Xw


## 4.1.2. Privacy Analysis of LDA-DP Algorithm

Theorem 5.
The within-class mean vectorμkk=1,2 in Algorithm 1 satisfies ε1,δ1 differential privacy when each entry of gkk=1,2 is sampled from N0,σ12, where σ1≥p3/2ln2/πδ12+p3ln2/πδ12+2ε1/ε1,0<δ1<2/π.Proof. We denote the two neighboring data sets areX=X1∪X2 and X^=X^1∪X^2, where only one individual is different, without losing general assumption. Suppose the different individuals are in X1 and X^1, we denote them as xN1≠x^N1, they are p -dimensional unit vector. We denote a=∑x∈X1x and a^=∑x∈X1x, let c=a+g1 and c^=a^+g^1, each entry of g1 and g^1 is sampled from N0,σ12.The log ratio of the probabilitiesc and c^ at a point h is lnPc=h|X/Pc^=h|X^, the numerator in the ratio describes the probability of seeing h when the data set is X, the denominator corresponds the probability of seeing this same value when the data set is X^.By Theorem1, we will to find the value of σ1 such that the inequality lnPc=h|X/Pc^=h|X^=lnPh−a|X/Ph−a^|X^≤ε1 holds at least with probability 1−δ1.(11)lnPh−a|XPh−a^|X^=12σ12∑i=1phi−a^i2−hi−ai2=12σ12∑i=1p2hi−aiai−a^i+ai−a^i2≤12σ12∑i=1p2hi−aixN1i−x^N1i+12σ12∑i=1pxN1i−x^N1i2.Using the Lagrange multiplier method, we can get the maximum value of the objective function∑i=1pxN1i+x^N1i is 2p under the condition of ∑i=1pxN1i2=1,∑i=1px^N1i2=1.Then, we can obtain:∑i=1pxN1i−x^N1i≤∑i=1pxN1i+x^N1i≤2p. Similarly, we can obtain the following:(12)∑i=1pxN1i−x^N1i2≤4.So,1/2σ12∑i=1p2hi−aixN1i−x^N1i≤2p3/2r/σ12, where gi1=hi−ai≤r, for all i, and 1/2σ12∑i=1pxN1i−x^N1i2≤2/σ12.Then,lnPb=h|X/Pb^=h|X^≤2p3/2r+2/σ12, this quantity is bounded by ε1 whenever gi1≤r≤ε1σ12−2/2p3/2.To ensure privacy loss bounded byε1 with probability at least 1−δ1, we require to find σ1 that satisfies this inequality Prgi1≥ε1σ12−2/2p3/2≤δ1, due to symmetry, we will find σ1 such that Prgi1≥ε1σ12−2/2p3/2≤δ1/2.The tail bound is as follows:(13)Prgi1>t=∫t+∞12πσ1e−x22σ12dx=x=t+y∫0+∞12πσ1e−t+y22σ12dy,≤12πσ1e−t22σ12∫0+∞e−tyσ12dy≤1tσ12πe−t22σ12.We lett=ε1σ12−2/2p3/2, then 1/tσ1/2πe−t2/2σ12≤δ1/2, then we obtain the following:(14)lntσ1+t22σ12≥ln22πδ1.Whenσ1≥2p3/2+4p3+8ε1/2ε1, the first term in (14) is non-negative. To make the inequality (14) hold, we let t2/2σ12=1/2σ12ε1σ12−2/2p3/22≥ln2/2πδ1, then we obtain the following:(15)σ1≥p3/2ln2/πδ12+p3ln2/πδ12+2ε1ε1,0<δ1<2π.Theorem 6.
The pooled within-class scatter matrixSw in Algorithm 1 satisfies ε2,δ2 differential privacy, when each entry in the symmetric random matrix G is sampled from N0,σ22, where (16)σ2≥p+1ln2πδ22+p+12ln2πδ22+4ε2/2ε2,0<δ2<2π.Proof. Two neighboring data sets areX=X1∪X2 and X^=X^1∪X^2, where only one entry is different, without losing general assumption, suppose the different entry are in X1 and X^1 and denoted them as xN1≠x^N1.Becauseμkk=1,2 in (5) satisfies differential privacy has been proved by Theorem 5 which can be treated as a constant in (5), so if we want to prove that this theorem holds, it is only necessary to prove that the first item ∑k=12∑x∈XkxxT in (5) satisfies ε2,δ2 differential privacy after adding random matrix G.We denoteB=∑k=12∑x∈XkxxT, B^=∑k=12∑x∈XkxxT, let C=B+G and C^=B^+G^, G and G^ are two independent symmetric random matrices with the upper triangle (including the diagonal) entries are sampled from N0,σ22, and make the symmetrical position entries in the lower triangle matrix equal to the upper triangle.The log ratio of the probabilitiesC and C^ at a point H is lnPC=H|X/PC^=H|X^.By Theorem1, we need to find the value of σ2 such that the inequality lnPC=H|X/PC^=H|X^≤ε2 holds at least with probability 1−δ2.(17)lnPC=HXPC^=HX^=lnPH−BXPH−B^X^=12σ22∑1≤i≤j≤pHij−B^ij2−Hij−Bij2=12σ22∑1≤i≤j≤p2Hij−BijBij−B^ij+Bij−B^ij2≤12σ22∑1≤i≤j≤p2Hij−BijxN1ixN1j−x^N1ix^N1j+12σ22∑1≤i≤j≤pxN1ixN1j−x^N1ix^N1j2.By using the Lagrange multiplier method and the inequality in [18], the following inequalities hold:(18)∑1≤i≤j≤pxN1ixN1j−x^N1ix^N1j2≤2,∑1≤i≤j≤pxN1ixN1j−x^N1ix^N1j≤p+1.Then,lnPH−BX/PH−B^|X^|≤rp+1+1/σ22, where Gij=Hij−Bij≤r for all i,j.The rest of the proof process is similar to Theorem5, then we can obtain the following:(19)σ2≥p+1ln2/πδ22+p+12ln2/πδ22+4ε22ε2,0<δ2<2π.We have proven that the within-class mean vectorμkk=1,2 satisfies ε1,δ1 differential privacy, the pooled within-class scatter matrix Sw satisfies ε2,δ2 differential privacy, by the property of differential privacy sequential composition, the projection direction vector in the Algorithm 1 satisfies ε,δ differential privacy, where ε=ε1+ε2,δ=δ1+δ2. For the published projection data X˜=Xw, X∈RN×p, w∈Rp×1,p<N, we can regard X˜=Xw as a set of undetermined system of equation, the number of variables are more than equations, so the equation has infinitely many sets of solutions, that is, it is impossible to infer the information of the original data X from the published projection data X˜.


## 4.2. Mul-LDA-DP Algorithm

In this section, we propose the Mul-LDA-DP algorithm for distributed data publishing. The mathematical notations used in this section are summarized in Table2.Table 2
Summary of notations.
NotationExplanationMThe number of data ownersPmThem-th data ownerNmkThe number of individuals in thek-th class owned by PmNkThe total number of individuals in thek -th class, Nk=∑m=1MNmkXmkThe set of thek -th class data owned by PmXkThe set of thek -th class data. Xk=∪m=1MXmkXmThe data set owned byPm. Xm=∪k=12XmkμmkThe within-class mean vector of thek -th class data owned by PmμkThe within-class mean vector of thek -th class dataSmkThe within-class scatter matrix of thek -th class data owned by PmSkThe within-class scatter matrix of thek-th class dataSwThe pooled within-class scatter matrix


### 4.2.1. Problem Statement and Algorithm Proposed

In the distributed scenario, data are stored by multiple data owners rather than a single owner, and the data owners do not trust each other. Data at a single site may not be sufficient for statistical learning. One solution is that each data owner uses the LDA-DP algorithm in Section4.1 to publish the projection data independently. Another solution is the data owners cooperate with each other to publish the projection data of the integrated data. Comparing the two solutions, it is obvious that the latter solution can improve the utility of publishing data. Based on the idea of the second solution and [32], we propose the Mul-LDA-DP algorithm for distributed data publishing. The entity description of the model is as follows.(1)
Data owner. The data ownerPmm=1,2,…,M has a data set Xm. Each data owner can generate random vectors and matrices to perturb the within-class mean vectors and within-class scatter matrices locally.(2)
Data publisher. The data publisher is a data publishing platform based on blockchain. The data publisher aggregates the local within-class mean vectors and within-class scatter matrices with noise. The data publisher can obtain the projection vector that satisfies differential privacy and publishes the projection data of the pooled data.(3)
A random number generator. It can generate random vectors and random matrices and send them to data owners and data publisher secretly.Threat Model. In our setting, we assume that the data owners and data publisher are honest-but-curious, that is, they follow the protocol but may try to deduce information of other data owners from the received messages.Two types of adversaries are considered, which are external attackers and internal attackers. External attackers which can be called an external eavesdropper may gain access to information such as data sent by data owners to the data publisher. Internal adversaries can be the data owners and the data publisher. The goal of each data owner is to extract the information not owned by him, while the goal of the data publisher is to extract the information from each data owner.Distributed Within-Class Mean Vectors and Pooled Within-Class Scatter Matrix Computation. When the data are owned byM data owners, the within-class mean vectors (1) can be decomposed into the following:(20)μk=1Nk∑m=1MNmkμmk,k=1,2,where μmk=1/Nmk∑x∈Xmkx.The pooled within-class scatter matrix (5) can be decomposed into the following:(21)Sw=∑k=12Sk=∑k=12∑m=1MSmk=∑m=1M∑k=12Smk,where Smk=∑x∈XmkxxT−NmkμmkμmkT.The abovementioned result allows each data owner to compute and perturb a partial result simultaneously locally. Therefore, we use the additivity of Gaussian distribution to propose a correlated noise generation scheme. We design the noise generation procedure such that (i) we can ensure that the data output from each data owner satisfy differential privacy and (ii) we can achieve the noise level of the same as the pooled data scenario.Scheme for Perturbing Shared Data by Correlated Noise. To prevent the data publisher and other data owners learning the privacy of local data, the data owner uses the noise generated by himself and the noise generated by the random number generator to perturb the local within-class mean vectors and within-class scatter matrices. Through our correlated noise design scheme, the data aggregated by the data publisher contain the same level of noise as the centralized scenario. The scheme is described as below:(1)
Initialization stage. The random number generator generatesp dimensional random vectors g¯mk, each entry is sampled from N0,M−1/Mσ12, generates p×p random matrices G¯m, let G¯m be the symmetric matrix with the upper triangle (including the diagonal) entries are sampled from N0,M−1/Mσ22, and makes the symmetrical position entries in the lower triangle matrix equal to the upper triangle, m=0,1,2,…,M,k=1,2. Make these random vectors and matrices satisfy ∑m=0Mg¯mk=0,∑m=0MG¯m=0p×p, then g¯mkk=1,2 and G¯m are sent to data owner Pm secretly, g¯0kk=1,2 and G¯0 are sent to the data publisher secretly.(2)
Data ownerPm generates p dimensional random vectors gmkk=1,2, each entry is sampled from N0,1/Mσ12, computes μmk=1/Nmk∑x∈Xmkx+gmk+g¯mk,k=1,2, and sends them to the data publisher.(3)
The data publisher computesμk=1/Nk∑m=1MNmkμmk+g¯0k,k=1,2 and sends them to each data owner.(4)
The data ownerPm generates p×p random matrix Gm, let Gm be the symmetric matrix with the upper triangle (including the diagonal) entries are sampled from N0,1/Mσ22, and make the symmetrical position entries in the lower triangle matrix equal to the upper triangle. Data owner Pm computes Sm=∑k=12Smk+Gm+G¯m and sends it to the data publisher.(5)
The data publisher computesSw=∑m=1MSm+G¯0 and calculates the projection vector w that satisfies differential privacy.The specific details of Mul-LDA-DP algorithm are in Algorithm 2. The input random vectorsg¯mk and random matrices G¯mk in Algorithm 2 are generated in the initialization stage by the random number generator, m=0,1,2,…,M,k=1,2.Algorithm 2:Mul-LDA-DP algorithm.
Input: Data sets Xm, m=1,2,…,M,k=1,2, privacy parameters ε1,δ1, ε2,δ2, random vector g¯mk and random matrix G¯m which are generated in initialization stage, m=0,1,2,…,M;k=1,2.Output: Projection direction vector w, projection data X˜(1)
form=1 to Mdo(2)
fork=1 to 2 do(3)
Setσ1=p3/2ln2/πδ12+p3ln2/πδ12+2ε1/ε1, data owner generates p dimensional random vector gmk, each entry is sampled from N0,σ12/M(4)
Computeμmk=1/Nmk∑x∈Xmkx+gmk+g¯mk(5)
end for(6)
end for(7)
Computeμk=1/Nk∑m=1MNmkμmk+g¯0k(8)
form=1 to M do(9)
Setσ2=p+1ln2/πδ22+p+12ln2/πδ22+4ε2/2ε2, data owner generates p×p symmetric random matrices Gm, each entry is sampled from N0,σ22/M(10)
fork=1 to 2 do(11)
ComputeSmk=∑x∈XmkxxT−NmkμmkμmkT(12)
end for(13)
ComputeSm=∑k=12Smk+Gm+G¯m(14)
end for(15)
ComputeSw=∑m=1MSm+G¯0(16)
Computew=Sw−1μ1−μ2(17)
returnX˜=∪m=1MXmw


### 4.2.2. Privacy Analysis of the Mul-LDA-DP Algorithm

Theorem 7.
The within-class mean vectorμkk=1,2 in Algorithm 2 satisfies ε1,δ1 differential privacy.Proof.μmk=1/Nmk∑x∈Xmkx+gmk+g¯mk because each entry of gmk is sampled from N0,1/Mσ12, and each entry of g¯mk is sampled from N0,M−1/Mσ12, so each entry of gmk+g¯mk obeys N0,σ12. By Theorem 5, μmk satisfies ε1,δ1 differential privacy.Due to the post-processing property of differential privacy, the within-class mean vectorμk=1/Nk∑m=1MNmkμmk+g¯0k in Algorithm 2 satisfies ε1,δ1 differential privacy.Theorem 8.
The pooled within-class scatter matrixSw in Algorithm 2 satisfies ε2,δ2 differential privacy.Proof.Sm=∑k=12Smk+Gm+G¯m , where each entry of symmetric random matrix Gm is sampled from N0,1/Mσ22, and each entry of symmetric random matrix G¯m is sampled from N0,M−1/Mσ22, so each entry of G¯m+G¯m obeys N0,σ22. By Theorem 6, Sm satisfies ε2,δ2 differential privacy. Due to the post-processing property of differential privacy, the pooled within-class scatter matrix Sw=∑m=1MSm+G¯0 in Algorithm 2 satisfies ε2,δ2 differential privacy.We have proven bothμkk=1,2 and Sw satisfy differential privacy, we will show that the level of noise is the same as the centralized scenario. In the initialization stage, the noise vectors and matrices generated by the random number generator satisfy ∑m=0Mg¯mk=0 and ∑m=0MG¯m=0p×p.The within-class mean vectorμkk=1,2 is as follows:(22)μk=1Nk∑m=1MNmkμmk+g¯0k=1Nk∑m=1M∑x∈Xmkx+gmk+g¯mk+g¯0k,=1Nk∑m=1M∑x∈Xmkx+∑m=1Mgmk+∑m=0Mg¯mk=1Nk∑m=1M∑x∈Xmkx+∑m=1Mgmk,=1Nk∑x∈Xkx+∑m=1Mgmk.Each entry of∑m=1Mgmk obeys N0,σ12.The pooled within-class scatter matrixSw is as follows:(23)Sw=∑m=1MSm+G¯0=∑m=1M∑k=12Smk+Gm+G¯m+G¯0,=∑m=1M∑k=12Smk+∑m=1MGm+∑m=0MG¯m=∑k=12∑m=1MSmk+∑m=1MGm,=∑k=12Sk+∑m=1MGm.Each entry of∑m=1MGm obeys N0,σ22.According to Theorems5 and 6, the within-class mean vector μkk=1,2 and pooled within-class scatter matrix Sw contain the same level of noise as the centralized scenario, and we achieve the purpose of improving the utility of publishing data while protecting the data privacy.There are three opportunities for attackers to steal the data transmitted between the data owner and the data publisher. The first time is that the data owner sends the within-class mean vectors to the data publisher, the second time is that the data owner sends the within-class scatter matrices to data publisher. From Theorems7 and 8, we know that the within-class mean vectors and the within-class scatter matrices satisfy differential privacy. Therefore, the attacker cannot infer the information of the original data from the eavesdropped data. The third time is that the data owner sends projection data to the data publisher, in Section 4.1.2, we have analyzed that it is impossible to infer the information of the original data from the published projection data.


## 4.2.1. Problem Statement and Algorithm Proposed

In the distributed scenario, data are stored by multiple data owners rather than a single owner, and the data owners do not trust each other. Data at a single site may not be sufficient for statistical learning. One solution is that each data owner uses the LDA-DP algorithm in Section4.1 to publish the projection data independently. Another solution is the data owners cooperate with each other to publish the projection data of the integrated data. Comparing the two solutions, it is obvious that the latter solution can improve the utility of publishing data. Based on the idea of the second solution and [32], we propose the Mul-LDA-DP algorithm for distributed data publishing. The entity description of the model is as follows.(1)
Data owner. The data ownerPmm=1,2,…,M has a data set Xm. Each data owner can generate random vectors and matrices to perturb the within-class mean vectors and within-class scatter matrices locally.(2)
Data publisher. The data publisher is a data publishing platform based on blockchain. The data publisher aggregates the local within-class mean vectors and within-class scatter matrices with noise. The data publisher can obtain the projection vector that satisfies differential privacy and publishes the projection data of the pooled data.(3)
A random number generator. It can generate random vectors and random matrices and send them to data owners and data publisher secretly.Threat Model. In our setting, we assume that the data owners and data publisher are honest-but-curious, that is, they follow the protocol but may try to deduce information of other data owners from the received messages.Two types of adversaries are considered, which are external attackers and internal attackers. External attackers which can be called an external eavesdropper may gain access to information such as data sent by data owners to the data publisher. Internal adversaries can be the data owners and the data publisher. The goal of each data owner is to extract the information not owned by him, while the goal of the data publisher is to extract the information from each data owner.Distributed Within-Class Mean Vectors and Pooled Within-Class Scatter Matrix Computation. When the data are owned byM data owners, the within-class mean vectors (1) can be decomposed into the following:(20)μk=1Nk∑m=1MNmkμmk,k=1,2,where μmk=1/Nmk∑x∈Xmkx.The pooled within-class scatter matrix (5) can be decomposed into the following:(21)Sw=∑k=12Sk=∑k=12∑m=1MSmk=∑m=1M∑k=12Smk,where Smk=∑x∈XmkxxT−NmkμmkμmkT.The abovementioned result allows each data owner to compute and perturb a partial result simultaneously locally. Therefore, we use the additivity of Gaussian distribution to propose a correlated noise generation scheme. We design the noise generation procedure such that (i) we can ensure that the data output from each data owner satisfy differential privacy and (ii) we can achieve the noise level of the same as the pooled data scenario.Scheme for Perturbing Shared Data by Correlated Noise. To prevent the data publisher and other data owners learning the privacy of local data, the data owner uses the noise generated by himself and the noise generated by the random number generator to perturb the local within-class mean vectors and within-class scatter matrices. Through our correlated noise design scheme, the data aggregated by the data publisher contain the same level of noise as the centralized scenario. The scheme is described as below:(1)
Initialization stage. The random number generator generatesp dimensional random vectors g¯mk, each entry is sampled from N0,M−1/Mσ12, generates p×p random matrices G¯m, let G¯m be the symmetric matrix with the upper triangle (including the diagonal) entries are sampled from N0,M−1/Mσ22, and makes the symmetrical position entries in the lower triangle matrix equal to the upper triangle, m=0,1,2,…,M,k=1,2. Make these random vectors and matrices satisfy ∑m=0Mg¯mk=0,∑m=0MG¯m=0p×p, then g¯mkk=1,2 and G¯m are sent to data owner Pm secretly, g¯0kk=1,2 and G¯0 are sent to the data publisher secretly.(2)
Data ownerPm generates p dimensional random vectors gmkk=1,2, each entry is sampled from N0,1/Mσ12, computes μmk=1/Nmk∑x∈Xmkx+gmk+g¯mk,k=1,2, and sends them to the data publisher.(3)
The data publisher computesμk=1/Nk∑m=1MNmkμmk+g¯0k,k=1,2 and sends them to each data owner.(4)
The data ownerPm generates p×p random matrix Gm, let Gm be the symmetric matrix with the upper triangle (including the diagonal) entries are sampled from N0,1/Mσ22, and make the symmetrical position entries in the lower triangle matrix equal to the upper triangle. Data owner Pm computes Sm=∑k=12Smk+Gm+G¯m and sends it to the data publisher.(5)
The data publisher computesSw=∑m=1MSm+G¯0 and calculates the projection vector w that satisfies differential privacy.The specific details of Mul-LDA-DP algorithm are in Algorithm 2. The input random vectorsg¯mk and random matrices G¯mk in Algorithm 2 are generated in the initialization stage by the random number generator, m=0,1,2,…,M,k=1,2.Algorithm 2:Mul-LDA-DP algorithm.
Input: Data sets Xm, m=1,2,…,M,k=1,2, privacy parameters ε1,δ1, ε2,δ2, random vector g¯mk and random matrix G¯m which are generated in initialization stage, m=0,1,2,…,M;k=1,2.Output: Projection direction vector w, projection data X˜(1)
form=1 to Mdo(2)
fork=1 to 2 do(3)
Setσ1=p3/2ln2/πδ12+p3ln2/πδ12+2ε1/ε1, data owner generates p dimensional random vector gmk, each entry is sampled from N0,σ12/M(4)
Computeμmk=1/Nmk∑x∈Xmkx+gmk+g¯mk(5)
end for(6)
end for(7)
Computeμk=1/Nk∑m=1MNmkμmk+g¯0k(8)
form=1 to M do(9)
Setσ2=p+1ln2/πδ22+p+12ln2/πδ22+4ε2/2ε2, data owner generates p×p symmetric random matrices Gm, each entry is sampled from N0,σ22/M(10)
fork=1 to 2 do(11)
ComputeSmk=∑x∈XmkxxT−NmkμmkμmkT(12)
end for(13)
ComputeSm=∑k=12Smk+Gm+G¯m(14)
end for(15)
ComputeSw=∑m=1MSm+G¯0(16)
Computew=Sw−1μ1−μ2(17)
returnX˜=∪m=1MXmw


## 4.2.2. Privacy Analysis of the Mul-LDA-DP Algorithm

Theorem 7.
The within-class mean vectorμkk=1,2 in Algorithm 2 satisfies ε1,δ1 differential privacy.Proof.μmk=1/Nmk∑x∈Xmkx+gmk+g¯mk because each entry of gmk is sampled from N0,1/Mσ12, and each entry of g¯mk is sampled from N0,M−1/Mσ12, so each entry of gmk+g¯mk obeys N0,σ12. By Theorem 5, μmk satisfies ε1,δ1 differential privacy.Due to the post-processing property of differential privacy, the within-class mean vectorμk=1/Nk∑m=1MNmkμmk+g¯0k in Algorithm 2 satisfies ε1,δ1 differential privacy.Theorem 8.
The pooled within-class scatter matrixSw in Algorithm 2 satisfies ε2,δ2 differential privacy.Proof.Sm=∑k=12Smk+Gm+G¯m , where each entry of symmetric random matrix Gm is sampled from N0,1/Mσ22, and each entry of symmetric random matrix G¯m is sampled from N0,M−1/Mσ22, so each entry of G¯m+G¯m obeys N0,σ22. By Theorem 6, Sm satisfies ε2,δ2 differential privacy. Due to the post-processing property of differential privacy, the pooled within-class scatter matrix Sw=∑m=1MSm+G¯0 in Algorithm 2 satisfies ε2,δ2 differential privacy.We have proven bothμkk=1,2 and Sw satisfy differential privacy, we will show that the level of noise is the same as the centralized scenario. In the initialization stage, the noise vectors and matrices generated by the random number generator satisfy ∑m=0Mg¯mk=0 and ∑m=0MG¯m=0p×p.The within-class mean vectorμkk=1,2 is as follows:(22)μk=1Nk∑m=1MNmkμmk+g¯0k=1Nk∑m=1M∑x∈Xmkx+gmk+g¯mk+g¯0k,=1Nk∑m=1M∑x∈Xmkx+∑m=1Mgmk+∑m=0Mg¯mk=1Nk∑m=1M∑x∈Xmkx+∑m=1Mgmk,=1Nk∑x∈Xkx+∑m=1Mgmk.Each entry of∑m=1Mgmk obeys N0,σ12.The pooled within-class scatter matrixSw is as follows:(23)Sw=∑m=1MSm+G¯0=∑m=1M∑k=12Smk+Gm+G¯m+G¯0,=∑m=1M∑k=12Smk+∑m=1MGm+∑m=0MG¯m=∑k=12∑m=1MSmk+∑m=1MGm,=∑k=12Sk+∑m=1MGm.Each entry of∑m=1MGm obeys N0,σ22.According to Theorems5 and 6, the within-class mean vector μkk=1,2 and pooled within-class scatter matrix Sw contain the same level of noise as the centralized scenario, and we achieve the purpose of improving the utility of publishing data while protecting the data privacy.There are three opportunities for attackers to steal the data transmitted between the data owner and the data publisher. The first time is that the data owner sends the within-class mean vectors to the data publisher, the second time is that the data owner sends the within-class scatter matrices to data publisher. From Theorems7 and 8, we know that the within-class mean vectors and the within-class scatter matrices satisfy differential privacy. Therefore, the attacker cannot infer the information of the original data from the eavesdropped data. The third time is that the data owner sends projection data to the data publisher, in Section 4.1.2, we have analyzed that it is impossible to infer the information of the original data from the published projection data.


## 5. Experiment

In order to measure the usability of the LDA-DP and Mul-LDA-DP algorithms proposed in this paper, we conduct experiments on real data sets which are Adult and NLTCS. Adult data set is extracted from the 1994 US Census, it contains 45222 individuals, each individual has 15 attributes. NLTCS data set is extracted from the National Long Term Care Survey, and recorded the daily activities of 21574 disabled persons at different time periods, each individual has 16 attributes. We use the SVM misclassification rate to measure the availability of the published data. For the Adult data set, it is necessary to predict whether a person (1) holds a post-secondary degree and (2) earns more than 50K. For the NLTCS data set, we need to predict whether a person (1) is unable to get outside, (2) is unable to manage money, (3) is unable to travel, and (4) is unable to bath. In our experiments, we setδ=0.001 to remain unchanged, and ε to take different values. We uniformly divide the privacy parameters into 2 portions ε1=ε2=ε/2,δ1=δ2=δ/2. Each experiment was repeated 50 times, and the mean value was taken as the experimental result. We use “No Privacy” to represent the SVM misclassification rate on the original data set.


### 5.1. Comparing the Performance of LDA-DA, PrivBayes, and PRivHD Algorithms under Different Privacy Budgets

The LDA-DA, PrivBayes, and PrivHD algorithms are all suitable for the centralized data publishing scenario, so in this set of experiments, we set the number of data owners to 1, and privacy budgetε takes different values. As can be seen from Figure 1, for both Adult and NLTCS data sets, the SVM classification utility of the data published by the LDA-DP algorithm outperforms the PrivBayes algorithm. The LDA-DP algorithm outperforms the PrivHD algorithm on the NLTCS dataset; however, the LDA-DP algorithm has slightly lower SVM classification utility on the Adult dataset than the PrivHD algorithm. We can also observe a commonality, for LDA-DA, PrivBayes, and PRivHD algorithms, the SVM misclassification rate decreases with the increase of the privacy budget ε. This phenomenon is consistent with the theory that as the privacy budget ε increases, privacy protection will weaken and the availability of data will increase.Figure 1
SVM misclassification rate of LDA-DP, PrivBayes and PRivHD under different privacy budgets. (a) NLTCS,Y = money. (b) NLTCS, Y = outside. (c) NLTCS, Y = bathing. (d) NLTCS, Y = travelling. (e) Adult, Y = salary. (f) Adult, Y = education.
(a)(b)(c)(d)(e)(f)


### 5.2. Comparing the Performance of Mul-LDA-DA and DP-SUBN3 Algorithms under Different Privacy Budgets

The algorithm Mul-LDA-DP proposed in this paper is suitable for the distributed data publishing scenario, so in this set of experiments, we set the number of data owners to 3, and privacy budgetε takes different values. We train classifiers on published data set to compare the efficacy of Mul-LDA-DA and DP-SUBN3 algorithms. From Figure 2, we can see that the SVM classification utility of the data published by the Mul-LDA-DP algorithm outperforms the DP-SUBN3 algorithm. Both on money of NLTCS and education of Adult classifiers, the misclassification rate of Mul-LDA-DA algorithm is significantly lower than the DP-SUBN3 algorithm especially.Figure 2
SVM misclassification rate of Mul-LDA-DA and DP-SUBN3 under different privacy budgets. (a) NLTCS, Y = money. (b) NLTCS, Y = outside (c) Adult, Y = salary. (d) Adult, Y = education.
(a)(b)(c)(d)


### 5.3. Comparing the Performance of Mul-LDA-DA and DP-SUBN3 Algorithms under Different Number of Data Owners

In this section, the experiment studied the relationship between SVM misclassification rate and the number of data owners. The number of data owners is set to 2, 4, 6, 8, 10, and the privacy budgetε is set to 0.2, We trained two classifiers, education classifier, and salary classifier on Adult data set. The results in Figure 3 show that the SVM misclassification rate of the Mul-LDA-DP algorithm remains stable with the change of the number of data owners. The reason is that we perturb the local shared data by generating correlated noise based on the additivity of the Gaussian distribution. This scheme ensures that the level of Gaussian noise added to the data in the distributed scenario is similar to the noise level in the centralized scenario. Therefore, as the number of data owners increases, the misclassification rate remains stable. The SVM misclassification rate of DP-SUBN3 algorithm decreases as the number of data owners increases. This is because as the number of data owners increases, the number of update iterations increases when constructing the Bayesian network, and the Bayesian network constructed is closer to the distribution of the original data. However, from Figure 3, we can see that the performance of Mul-LDA-DA algorithm is still better than DP-SUBN3 algorithm when the number of data owners is no more than 10.Figure 3
SVM misclassification rate of Mul-LDA-D and DP-SUBN3 under different number of data owners. (a) Adult, Y = salary. (b) Adult, Y = education.
(a)(b)


## 5.1. Comparing the Performance of LDA-DA, PrivBayes, and PRivHD Algorithms under Different Privacy Budgets

The LDA-DA, PrivBayes, and PrivHD algorithms are all suitable for the centralized data publishing scenario, so in this set of experiments, we set the number of data owners to 1, and privacy budgetε takes different values. As can be seen from Figure 1, for both Adult and NLTCS data sets, the SVM classification utility of the data published by the LDA-DP algorithm outperforms the PrivBayes algorithm. The LDA-DP algorithm outperforms the PrivHD algorithm on the NLTCS dataset; however, the LDA-DP algorithm has slightly lower SVM classification utility on the Adult dataset than the PrivHD algorithm. We can also observe a commonality, for LDA-DA, PrivBayes, and PRivHD algorithms, the SVM misclassification rate decreases with the increase of the privacy budget ε. This phenomenon is consistent with the theory that as the privacy budget ε increases, privacy protection will weaken and the availability of data will increase.Figure 1
SVM misclassification rate of LDA-DP, PrivBayes and PRivHD under different privacy budgets. (a) NLTCS,Y = money. (b) NLTCS, Y = outside. (c) NLTCS, Y = bathing. (d) NLTCS, Y = travelling. (e) Adult, Y = salary. (f) Adult, Y = education.
(a)(b)(c)(d)(e)(f)


## 5.2. Comparing the Performance of Mul-LDA-DA and DP-SUBN3 Algorithms under Different Privacy Budgets

The algorithm Mul-LDA-DP proposed in this paper is suitable for the distributed data publishing scenario, so in this set of experiments, we set the number of data owners to 3, and privacy budgetε takes different values. We train classifiers on published data set to compare the efficacy of Mul-LDA-DA and DP-SUBN3 algorithms. From Figure 2, we can see that the SVM classification utility of the data published by the Mul-LDA-DP algorithm outperforms the DP-SUBN3 algorithm. Both on money of NLTCS and education of Adult classifiers, the misclassification rate of Mul-LDA-DA algorithm is significantly lower than the DP-SUBN3 algorithm especially.Figure 2
SVM misclassification rate of Mul-LDA-DA and DP-SUBN3 under different privacy budgets. (a) NLTCS, Y = money. (b) NLTCS, Y = outside (c) Adult, Y = salary. (d) Adult, Y = education.
(a)(b)(c)(d)


## 5.3. Comparing the Performance of Mul-LDA-DA and DP-SUBN3 Algorithms under Different Number of Data Owners

In this section, the experiment studied the relationship between SVM misclassification rate and the number of data owners. The number of data owners is set to 2, 4, 6, 8, 10, and the privacy budgetε is set to 0.2, We trained two classifiers, education classifier, and salary classifier on Adult data set. The results in Figure 3 show that the SVM misclassification rate of the Mul-LDA-DP algorithm remains stable with the change of the number of data owners. The reason is that we perturb the local shared data by generating correlated noise based on the additivity of the Gaussian distribution. This scheme ensures that the level of Gaussian noise added to the data in the distributed scenario is similar to the noise level in the centralized scenario. Therefore, as the number of data owners increases, the misclassification rate remains stable. The SVM misclassification rate of DP-SUBN3 algorithm decreases as the number of data owners increases. This is because as the number of data owners increases, the number of update iterations increases when constructing the Bayesian network, and the Bayesian network constructed is closer to the distribution of the original data. However, from Figure 3, we can see that the performance of Mul-LDA-DA algorithm is still better than DP-SUBN3 algorithm when the number of data owners is no more than 10.Figure 3
SVM misclassification rate of Mul-LDA-D and DP-SUBN3 under different number of data owners. (a) Adult, Y = salary. (b) Adult, Y = education.
(a)(b)


## 6. Conclusion

In this paper, we propose two algorithms for privacy preserving data publishing, the LDA-DP algorithm for data publishing in the scenario, and the Mul-LDA-DP algorithm for multiparty horizontally split data publishing. We use the additivity of Gaussian distribution to alleviate the effects of noise and can achieve the same noise level as the centralized scenario. The experimental results show that the projection data released by the two algorithms can maintain high utility in SVM classification. However, the research in this paper also has limitations. 1)We only research the privacy protection problem when the data are a binary classification, but they are often multiclassification data. 2)The data released by the two algorithms in this paper are low-dimensional projection data of the original data, which limit the analysis and mining of the released data in many aspects. In the future, we will continue to conduct research on the abovementioned issues.



---

*Source: 5612794-2022-05-09.xml*