Accuracy and Privacy Evaluations of Collaborative Data Analysis
Akira Imakura, Anna Bogdanova, Takaya Yamazoe, Kazumasa Omote, Tetsuya Sakurai.
1-1-1 Tennodai, Ibaraki, Tsukuba 305-8573, University of Tsukuba, Japan,
imakura@cs.tsukuba.ac.jp, bogdanova.anna.fw@u.tsukuba.ac.jp, s1920600@u.tsukuba.ac.jp, omote@risk.tsukuba.ac.jp,
sakurai@cs.tsukuba.ac.jp.

Abstract
Distributed data analysis without revealing the individual data
has recently attracted significant attention in several applications. A collaborative data analysis through sharing dimensionality reduced representations of data has been proposed
as a non-model sharing-type federated learning. This paper
analyzes the accuracy and privacy evaluations of this novel
framework. In the accuracy analysis, we provided sufficientconditions for the equivalence of the collaborative data analysis and the centralized analysis with dimensionality reduction. In the privacy analysis, we proved that collaborative
users’ private datasets are protected with a double privacy
layer against insider and external attacking scenarios.

Introduction
Background
Recently, the problem of real-life data availability for machine learning and data analysis applications came to theforefront of actual research challenges. In particular, usecases that pertain to sensitive personal information or corporate secrecy can benefit from the ability to process distributed data without revealing it to other parties.
Various methods have been proposed over recent years,
involving sharing a machine learning model that is collectively trained among several parties. In the present work,
we analyze an alternative method of distributed and privacypreserving data analysis that does not require sharing themachine learning model. The non-model-sharing approach
has certain advantages over model-sharing methods: (a)maintaining the secrecy of particular model architecture; (b)protection from model poisoning attacks; (c) avoiding iterative communications necessary for machine learning training; (d) option to outsource data analysis to a third party ora data analysis competition.
The collaborative data analysis considered in this paper had been previously proposed in (Imakura and Sakurai 2020; Imakura, Ye, and Sakurai 2020). However, themethod’s proper accuracy and privacy analysis were lacking.
In present work, we fill in this gap by providing conditions
for equivalence of the data analysis in centralized and distributed settings, as well as conducting a thorough privacy
Copyright c 2021, Association for the Advancement of Artificial
Intelligence (www.aaai.org). All rights reserved.

analysis and disclosure risk evaluation of the collaborative
data analysis.

Main purposes and contributions
The accuracy and privacy analyses are essential in practicaluse of the collaborative data analysis. In this paper, we analyze the equivalence of the collaborative data analysis andthe centralized analysis with dimensionality reduction. We
also analyze the privacy of the private dataset against insider
and external attacking scenarios.
The main contributions of this paper are summarized asfollows:• We provided the sufficient condition for equivalence ofthe collaborative data analysis and the centralized analysis
with dimensionality reduction.
• We proved that, in the collaborative data analysis, the private dataset is protected based on a double privacy layer
against insider and external attacking scenarios.
• We demonstrated numerical evaluations for the accuracy
and privacy analyses.

Related Work
The problem of deriving insights from data while maintaining the privacy of individual data records was first addressedin the context of Data Mining and Knowledge Discovery in
Databases (KDD) (Agrawal and Srikant 2000) and consequently formed a large body of literature known as PrivacyPreserving Data Mining (PPDM). This field studies datasanitizing operations, which can offer quantifiable privacy
guarantees, while maintaining data utility for a variety ofdownstream analytical tasks, including supervised and unsupervised machine learning (Mendes and Vilela 2017).
As there are multiple definitions of what constitutes privacy and how it should be measured, distinct privacy guarantees and methods of privacy production became known
in the literature as privacy models. Most influential privacy
models that emerged from PPDM are k-anonymity, proposed by Samarati and Sweeny (1998), and ε-differential
privacy, introduced by Dwork (2008). K-anonymity protects
users’ data from linkage attacks, ensuring that released data
has at least k identical records. ε-differential privacy, on theother hand, guarantees that the inclusion of any record inthe dataset will not change the output of data mining operations by more than ε, thus preventing membership inference attacks. Both privacy models are theoretically sound,
deployed in practice, and legislatively recognized. However,there are significant shortcomings that call for the development of alternative notions of privacy. Thus, k-anonymity isproven to be NP-hard (Verykios et al. 2004) and not attainable on high dimensional and sparse datasets (Narayanan
and Shmatikov 2006). Similarly, providing record-level εdifferential privacy is not suitable for modern deep learning
applications (Zhao, Chen, and Zhang 2019), as the amountof perturbation required diminishes data utility.
With the advancement of highly parameterized machinelearning models, the focus of data privacy research shifted
towards designing model architectures and sanitizing model
parameters to enable Privacy-Preserving Machine Learning (PPML). One particularly successful approach had been
Federated Learning proposed by McMahan et al. (2017).
It is a machine learning framework that allows distributed
training of deep learning models through the averaging of
gradient descent steps taken on private datasets. Analogous
algorithms were introduced for numerous machine learningmodels and various distributed settings, forming what became known as Federated Learning Systems (Li, Wen, andHe 2019). Although currently, PPML cannot provide a formal privacy guarantee, as PPDM does, it satisfies privacy
requirements through the data minimization approach, bysharing only the information necessary to the particular analytical task (Kairouz et al. 2019). Additionally, PPML is often combined with encryption schemes to prevent inference
from intermediary results of the computation.
The collaborative data analysis (Imakura and Sakurai
2020; Imakura, Ye, and Sakurai 2020), the method considered in this paper, is positioned in between the two approaches to privacy-preserving data analysis. It shares with
PPDM the focus on data transformation and the release ofsample-vise information, which can be further explored for
hidden relations and patterns. At the same time, it employs
the information-minimization approach of PPML throughthe dimensionality reduction operation on the original data.
Moreover, the shared intermediate representations can beformed by hidden layers of deep neural networks, strongly
relating our method to the PPML domain.
Privacy-preserving properties of dimensionality reduction
were previously explored in several papers. Thus, Tai etel. (2018), demonstrated that dimensionality reduction on
average increases the value of k in k-anonymity privacy
model, although it does not perform it reliably. Similarly,
Nguyen and colleagues (2020) developed a ε-DR privacy
framework of measuring the information loss of dimensionality reduction operations analogous to ε-differential
privacy. Formal privacy guaranteed were demonstrated forparticular methods of dimensionality reduction, such as
non-metric multidimensional scaling (MDS) (Alotaibi et al.
2012) and random projections (Liu, Kargupta, and Ryan
2005). Moreover, specialized methods of dimensionality reduction were developed to satisfy certain privacy models, forinstance differential-private Principal Component Analysis,
and differential-private Linear Discriminant Analysis (Jiang

et al. 2013). Since Data Collaboration method assumes anarbitrary dimensionality reduction function applied at theuser’s side, in practical applications such methods can bechosen to satisfy necessary privacy standards.
To the best of our knowledge, Data Collaboration is
the only method so far offering collaborative data analysis
through sharing dimensionality reduced representations ofdata and integration of such representations in a unified subspace. In this work, for the first time we propose privacy
guarantees as well as utility measures of the transformed
collaborative representations of data.

Collaborative data analysis
Distributed data analysis
Let m and n denote the numbers of features and training data
samples. In addition, let X = [x1 , x2 , . . . , xn ]T ∈ Rn×m
and Y = [y1 , y2 , . . . , yn ]T ∈ Rn×` be the training dataset
and the corresponding ground truth. The n data samples are
partitioned into c parties as follows:




X1
Y1
 X2 
 Y2 



X=
(1)
 ...  , Y =  ...  .
Xc

Yc

Then, the i-th party has partial dataset and the corresponding
ground truth,
Xi ∈ Rni ×m ,Yi ∈ Rni ×` .

A motivating example could be found in distributed medical data analysis. An analysis only using the dataset in each
medical institution, i.e., individual analysis may not be sufficient for generating a high-quality prediction result due toinsufficiency and imbalance of the data samples. If we cancentralize the datasets from multiple institutions and analyze
them as one dataset, i.e., centralized analysis, then we expect
to achieve a high-quality prediction. However, it is difficult
to centralise the original medical data samples with thosefrom other institutions due to confidentiality concerns. Suchkind of distributed data analysis is also essential in other applications, e.g., financial and manufacturing data analysis.

Outline of the collaborative data analysis
The collaborative data analysis has been proposed in(Imakura and Sakurai 2020; Imakura, Ye, and Sakurai 2020)
as a method of distributed data analysis. A practical operation strategy regarding privacy and confidentiality concerns
is also introduced. Here, we briefly introduce the algorithm
based on the practical operation strategy.
In the practical operation strategy, the collaborative data
analysis is operated by two roles: user and analyst. The users
have the private dataset Xi and the corresponding groundtruth Yi and want to analyze them without sharing Xi . Each
user individually constructs a dimensionality reduced intermediate representation and centralize it to analyst. To allow
each user to use individual function for generating the intermediate representation, analyst transforms again the centralized intermediate representations to an incorporable form

called collaboration representations. For constructing theincorporable collaboration representations, users generate ashareable anchor dataset and centralize its intermediate representation to analyst. Then, the collaborative representation
is analyzed as one dataset.
Training phase First, all users generate the same anchor
dataset X anc ∈ Rr×m , which is a shareable data consisting of public data or dummy data randomly constructed, andpartition it by features. Then, each user constructs the intermediate representations,ei
ei = fi (Xi ) ∈ Rni ×mX,eieianc = fi (X anc ) ∈ Rr×m
X,

where fi denotes a linear or nonlinear row-wise mapping
function and centralize the intermediate representations to
the analyst. A typical setting for fi is a dimensionality reduction, with me i < m, including unsupervised methods (Pearson 1901; He and Niyogi 2004; Maaten and Hinton 2008)
and supervised methods (Fisher 1936; Sugiyama 2007; Liet al. 2017; Imakura et al. 2019). For privacy and confidentiality concerns, the function fi should be set as• The private data Xi can be obtained only if anyone has
eiboth the corresponding intermediate representation X
and the mapping function fi or its approximation.
• The mapping function fi can be inferred only if anyone
has both the input and output of fi .
At the analyst side, the mapping function gi for the collaboration representation is constructed satisfyingbbianc = gi (Xe anc ) ∈ Rr×m
Xib anc ≈ Xb ancs.t. Xii0

0(i 6= i ),Here, for simplicity, we assume that the dimensionality of
ei does not depend on i. Also, the matrices Gi are computedX
as introduced in (Imakura and Sakurai 2020; Imakura, Ye,and Sakurai 2020), that is,cX
eianc Gi k2F ,minkZ − X(2)
f i ×cm
Gi ∈Rmi=1

r×m
ewhere Z ∈ R
is set as a column orthogonal matrix
whose columns are the left singular vectors corresponding
to the me largest singular values of a matrix
e1anc , X
e2anc , . . . , X
ecanc ] = X anc [F1 , F2 , . . . , Fc ].
[X

Theoretical evaluation for accuracy
In this paper, we analyze the accuracy of the collaborative
data analysis compared with the centralized analysis withedimensionality reduction B ∈ Rm×m
based on the norm
 2
X1 F1 G1
 X2 F2 G2  /kXk2F
XB − ..

.
Xc Fc Gc=c
XF

kXi B − Xi Fi Gi k2F /kXk2F .

(3)

i=1

With the anchor dataset X anc preserving statistics of X, weevaluate the accuracy (3) byc
X

kXi B − Xi Fi Gi k2F /kXk2F

i=1

in some sense. For computing gi , authors of (Imakura and
Sakurai 2020; Imakura, Ye, and Sakurai 2020) introduced apractical method via a total least squares problem when gi islinear and also indicated an idea when gi is nonlinear.
bi =Then, the obtained collaboration representations X
egi (Xi ) can be analyzed as one dataset,bb = [X
b1T , X
b2T , . . . , X
bcT ]T ∈ Rn×m
X,with the shared ground truth Yi using some supervised machine learning and the deep learning methods. The functions
gi and h are returned to the i-th user.
Prediction Phase Let Xitest ∈ Rsi ×m be a test datasetof the i-th party. Then, for prediction phase, the predictive
result Yitest of Xitest is obtained byYitest = h(gi (fi (Xitest )))via the intermediate and collaboration representations.

Accuracy analysiscX

kX anc B − X anc Fi Gi k2F /(ckX anc k2F )i=1
Pc
kX anc B − Zk2F + kZ − X anc Fi Gi k2F.
≤ i=1
ckX anc k2Fei =Under the assumption that fi are linear, we have X
eXi Fi , where Xi ∈ Rni ×m and Fi ∈ Rm×m
. Let F =[F1 , F2 , . . . , Fc ] and T 
Σ1
V1
anc
T
X F = U ΣV = [U1 , U2 ]
,
Σ2
V2T

 T 
ΣF 1
VF 1
F = UF ΣF VFT = [UF 1 , UF 2 ]
ΣF 2
VFT2
be singular value decompositions of matrices X anc F ande me
F . Here, Σ1 , ΣF 1 ∈ Rm×
are the diagonal matrices corresponding to me largest singular values. Note that Z = U1 .
Then, we havee 2kΣ2 k2 =min kX anc F − Xk
F

We analyze the accuracy of the collaborative data analysis
for the simple case that the mapping functions fi and gi arelinear, i.e.,
ei = fi (Xi ) = Xi Fi ,X
bi = gi (Xei ) = X
ei Gi ,X≈e mrank(X)=eF≤ kX anc F − X anc UF 1 ΣF 1 VFT1 k2F≤ kX anc k2F kF − UF 1 ΣF 1 VFT1 k2Fe
Fi ∈ Rm×m(rank(Fi ) = m),e= kX anc k2F kUF 2 ΣF 2 VFT2 k2Fe me
Gi ∈ Rm×(rank(Gi ) = m).
e= kX anc k2F kΣF 2 k2F .

Algorithm 1 Collaborative data analysis
Input (for user side): Xi ∈ Rni ×m , Yi ∈ Rni ×` , Xitest , individually
Output (for user side): Yitest (i = 1, 2, . . . , c).
user side (i)analyst side

1:
2:
3:
4:
5:
6:
7:
8:
9:
10:

———- Training phase ———Generate Xianc and share to all users
Set X anc
Generate fi
ei = fi (Xi ) and X
e anc = fi (X anc )Compute Xianceeei , Xe anc and Yi for all i
Share Xi , Xi and Yi to analyst→ Get Xie anc for all iConstruct gi from X
ibi = gi (Xei ) for all i
Compute X
b and Y
Set X
b and get h as Y ≈ h(X)bAnalyze X
Get gi and h← Return gi and h to user

11:———- Prediction phase ———Compute Yitest = h(gi (fi (Xitest )))Using this inequality, the norm (2) can be bounded by
min

c
X

Gi

=
≤

kZ − X anc Fi Gi k2F

i=1
c
X
i=1
c
X

min kZ − X anc Fi Gi k2FGianc
kGi k2F min
kZG−1
Fi k2F
i −X
−1
Gi

i=1





−1
−1
≤ max kGi k2F minkZ[G−1
1 , G2 , . . . , Gc ]
−1
i

Gi

− X anc [F1 , F2 , . . . , Fc ]k2F= max kGi k2F kΣ2 k2Fi≤ max kGi k2F kX anc k2F kΣF 2 k2F .
i

Therefore, the accuracy of the collaborative data analysis (3)can be evaluated by kΣ2 k2F and kΣF 2 k2F . Note that the valuekΣ2 k2F can be obtained at the analyst side.
This bound provides the following theorem.
Theorem 1. If the mapping functions satisfy
rank(X anc Fi ) = m,e(4)
the predictive results of the collaborative data analysis ismathematically equivalent to that of the centralized analysis
with dimensionality reduction F1 G1 .
R(F1 ) = R(F2 ) = · · · = R(Fc ),Proof. The condition (4) provides ΣF 2 = O, then we haveF1 G1 = F2 G2 = · · · = Fc Gc ,that proves the theorem.

Numerical evaluation for accuracy
Here, we provide numerical evaluation of the accuracy analysis. We used a 10-class classification of handwritten digits(MNIST) (LeCun 1998), where m = 784. We set the number of parties as c = 4 and the number of samples for eachparty as ni = 50.
Let B ∈ R784×25 be a mapping function generated by
PCA using all the training dataset X. We then set(1)

Fi = BEi(1)(2)+ εkBkF Ei ,i = 1, 2, . . . , c,(2)

with Ei ∈ R25×25 and Ei ∈ R784×25 whose entries
are normally distributed random numbers. We used a kernel
version of ridge regression (K-RR) (Saunders, Gammerman,
and Vovk 1998) with a Gaussian kernel for analyzing thecollaboration representation. The bandwidth σ of the Gaussian kernel is set based on the local scaling (Zelnik-Manorand Perona 2005). We set the regularization parameter for
K-RR to λ = 0.1. The anchor data X anc is constructed asa random matrix and r = 2, 000. Then, we evaluate the following four values,
τ1 = kΣ2 kF /kΣ1 kF ,
τ2 = kΣF 2 kF /kΣF kF ,
X1 F1 G1
 X2 F2 G2 

τ3 = XF1 G1 − ..
.
Xc Fc Gc

/kXF1 G1 kF ,
F

test
test
τ4 = 1 − NMI(YCDA
, YCA
),
test
test
where, in τ4 , the value NMI(YCDA, YCA) ∈ [0, 1] denotes
the normalized mutual information (NMI) between the prediction results of test dataset X test of the collaborative data
analysis and the centralized analysis with dimensionality reduction F1 G1 .

Figure 2: Scatter plot of log(τ4 ) v.s. log(τ1 ) and its roughbound log(τ1 )/2 + 0.5.

Figure 1: Scatter plot matrix and correlation coefficients forlog(τi ).

Note that, in order to obtain (approximately) the same predictive results as the centralized analysis with dimensionality reduction, we do not need to use the same mapping functions fi , but use different functions with (approximately) thesame range space.

Privacy analysis
All the numerical experiments were performed on Windows 10 Pro, Intel(R) Core(TM) i7-10710U CPU @1.10GHz, 16GB RAM using MATLAB2019b.
First, we perform the methods 10 times with ε = 0, thatis R(Fi ) = R(Fi0 ), but Fi 6= Fi0 for i 6= i0 . The obtained
average values areτ1 = 8.42 × 10−16 ,τ2 = 2.24 × 10−16 ,τ3 = 1.44 × 10−13 ,τ4 = 0.00,

that mean the collaborative data analysis obtains the same
result as the centralized analysis with the dimensionality retest
test
duction, i.e., YCDA
= YCA
. This result supports Theorem 1.
Next, we perform the methods 500 times with randomε ∈ [10−2 , 10−6 ] and evaluate correlation coefficients oflog(τi ). Figure 1 shows the scatter plot matrix and correlation coefficients for log(τi ), which demonstrates that theaccuracy of the collaborative data analysis regarding τ4 has
a strong correlation between τ1 , τ2 and τ3 . Therefore, fromthis result, we observed that the accuracy of the collaborative data analysis τ4 can be evaluated well by τ1 in practice.
Note that τ1 can be obtained at the analyst side.
We can also observed from Figure 2 that τ4 is roughly
bounded by
√log(τ1 )⇔ log(τ4 ) ≤+ log(c)τ4 ≤ c τ12with some constant c. Note that c = exp(0.5) in Figure 2.

Remarks on accuracy analysis
From the above analysis, we observed that
• If Fi satisfy the sufficient condition (4), the collaborative
data analysis achieves the same result of the centralized
analysis with dimensionality reduction.
• The accuracy of the collaborative data analysis comparedwith the centralized analysis with dimensionality reduction can be evaluated by kΣ2 kF in practice.

For the analysis, this paper considers the privacy of the private data Xi of each user in the collaborative data analysis.
Note that any information of the test data Xitest does nothave to be shared to others; see Algorithm 1.
Attacks for the data Xi can be classified into (i) attacks
to infer the characteristics of the training data; (ii) attacks
to infer the training data Xi itself; and (iii) attacks to infer
whether a data sample is in the training dataset or not, socalled the membership inference attack.
This paper considers the privacy of the data Xi itself,
rather than the characteristics of the data. We also shortly
discuss the privacy against the membership inference attack.

Privacy definitions of dimensionality reduction:
ε-DR privacy
Here, we introduce two dimensionality reduction (DR) privacy definitions: ε-DR privacy introduced in (Nguyen et al.
2019) and its variant, to evaluate the degree to which privacy is preserved through dimensionality reduction. Letee ∈ Rmf : x ∈ Rm → x(m > m)e be a dimensionality reduced function and f † be a reconstruction function of
f . Then, we evaluate degree of privacy preservation using
dist(x, x0 ) with a certain distance measure dist(·, ·), wherex0 = f † (f (x)).
The function f satisfies the ε-DR privacy regarding the
expected value, if we haveE[dist(x, x0 )] ≥ ε1(5)for each i.i.d. input sample x. The value ε1 depends on f
and is always larger than 0 with m > m.
e
Also, a function f satisfies the ε-DR privacy regarding asample set X = {x1 , x2 , . . . , xn }, if we havemin dist(x, x0 ) ≥ ε2 .

x∈X

(6)

The value ε2 depends on f and a set of samples X . Therefore, since ε depends on X , ε2 is possible to be 0 even if
m > m.
eAttacking scenarios
In this paper, we consider the following two attacking scenarios: insider and external attacks.
• Insider attacks. Here, the users and analyst will strictly
follow the strategy, but they try to infer the private data
Xi .
• External attacks. Here, we consider a man-in-the-middle
attack scenario where an attacker eavesdrops the information exchanged among users and analyst and infer theprivate data Xi .

Privacy against the honest-but-curious analyst
Theorem 2. For the collaborative data analysis, an honestbut-curious analyst cannot infer the private dataset Xi ofthe users, unless analyst does not collude with user(s).
Proof. For a privacy of Xi against the honest-but-curiouseianalyst, each user shares the intermediate representations X
anceand Xi to analyst. Here, we consider the possibility ofei and X
e anc .
recovering Xi from Xi
If analyst has a mapping function fi , analyst can infer Xi
by solving
ei = fi (Xi ).
X
However, the function fi is private in the collaborative data
analysis and also cannot be inferred by analyst, because analyst only has the output of fi , that is the intermediate representations, but has no input data of fi , if analyst does notcollude with user(s). Note that the function fi is constructedby some dimensionality reduction method with the private
data Xi . The function fi depends on Xi , so even if the dimensionality reduction method is identified, fi itself cannot
be inferred.
Thus, analyst cannot obtain the private data Xi from theintermediate representations, that proves the theorem.

Privacy against the honest-but-curious users
Theorem 3. For the collaborative data analysis, an honestbut-curious users cannot infer the private dataset Xi of aparticular user against collusion of up to c − 2 users.
Proof. For a privacy of Xi against other users, each user
shares the local anchor data Xianc to other users. Also, eachuser obtain the functions gi and h from analyst that is constructed based on information of private data of other users.
First, we consider the possibility of recovering the private data from Xianc . The local anchor data does not contain the original Xi , but may preserve some useful information. Here, the local anchor data is constructed by users
themselves using e.g., GAN and autoencoder with data
augmentation. Users can control the containing information although it may have a trade-off relationship between
the performance. Note that the collaborative data analysis
works well even using random anchor data as demonstratedin (Imakura and Sakurai 2020; Imakura, Ye, and Sakurai2020). Therefore, users cannot obtain the private information of Xi from Xianc .

Next, we consider the possibility of recovering the private data from gi and h. When c − 1 users i 6= i0 , where
the total number of users is c, are malicious and they collude together to retrieve information of the private dataset ofthe remaining (victim) user i0 , the colluding users have thefunction h and all Xi , fi , gi (i 6= i0 ). In this case, the function gi (i 6= i0 ) and h are constructed by Xi , fi (i 6= i0 ) of
the colluding users and Xi0 of the victim user. Therefore, theprivate data Xi0 of the victim user will be inferred by solving an inverse problem. On the other hand, when the c − 2users collude, the obtained functions gi and h of the colluding users are affected by remaining two users with equal importance. Therefore, users cannot infer each private dataset
Xi of the victim users.
Thus, an honest-but-curious users cannot infer the private
dataset of a particular user against collusion of up to c − 2users, that proves the theorem.

Privacy against collision of analyst with users
Theorem 4. If user(s) and analyst collude in the collaborative data analysis, the privacy of Xi is preserved regarding
ε-DR privacy definitions (5) and (6) of each fi .
Proof. If user(s) and analyst collude, then they can obtaine anc of fi . In this case, theyboth the input X anc and output Xi
can infer fi satisfying
eianc = fi (X anc ).
X
ei .
Therefore, using the inferred fi , they can infer Xi from X
However, since fi is a dimensionality reduced function, thatis m > me i , the privacy is still preserved regarding ε-DR
privacy definitions (5) and (6) of fi . In other words, the exact
ei , even using fi .
data Xi cannot be recovered from X

Privacy against the external attacks
Using secure data transmission protocols such as Transport
Layer Security (TLS), in which the transferred information is encrypted using the private key of the involving parties, the collaborative data analysis also protects the private
dataset Xi against the man-at-the-middle attackers. Note
that, in this case, we do not use secure multi-party computations, but just use encrypted communication for non private
data.
In the case that we do not use secure data transmission
protocols, the situation is almost the same as the case thatusers and analyst collude. That is, man-at-the-middle attacke anc and can infer Xi ; howers can infer fi from X anc and X
iever, the privacy is still preserved regarding ε-DR privacy
definitions (5) and (6) of fi .

Numerical evaluation for privacy analysis
Here, we provide numerical evaluation of the worst-case privacy analysis, i.e., the situation of Theorem 4. Let Bi ∈Rm×mi be a matrix for dimensionality reduction for Xi as
ei = Xi Bi ,X

Bi ∈ Rm×mi .

Table 1: Trade-off relationship between ε-DR privacy andprediction accuracy.
down-samplingminparameter εε-DR (7)0.0
7.36 × 10−6
0.00012.07 × 10−4
0.001
1.01 × 10−3
0.01
1.00 × 10−2
0.1
1.00 × 10−1
0.2
2.00 × 10−1
0.3
3.00 × 10−1
0.4
4.00 × 10−1
0.5
5.00 × 10−1
Centralized analysis
Individual analysis

(i)

(i)

Ave.
# of samples
100.00
99.97
99.75
97.50
76.85
56.43
39.35
25.84
15.98
1000.00
100.00

Ave.
ACC
92.8
92.8
92.8
92.8
91.7
90.3
88.7
86.1
81.4
93.6
75.5

(i)

Let Xi = [x1 , x2 , . . . , xni ]T . Then, if Bi and the centerµi ∈ Rm of dataset Xi are stolen, Xi is inferred by
0

0

0
(i)
(i)
†
T
T
e †
Xi0 = [x1 , x2 , . . . , x(i)ni ] = Xi Bi + 1µi (I − Bi Bi ),where B † is the pseudo-inverse of B and 1 = [1, 1, . . . , 1]T .
As a ε-DR privacy regarding a sample set (6), we set
(i)0

(i)

0

min dist(x, x ) = min

x∈X

i,j

kxj − xj k2(i)kxj k2.

(7)

Then, we use a down-sampling technique which removes
training data samples satisfying (7) < ε by changing ε andevaluate a trade-off relationship between (7) and a prediction
accuracy of the collaborative data analysis.
We used MNIST again. The dimensionality reduction matrix Bi is constructed by PCA using each Xi . We set c =10, ni = 100 and mi = 25 for parameters. Other settings ofnumerical evaluation are the same as used in the numericalevaluation for accuracy analysis.
Table 1 shows the trade-off relationship between a minimum ε-DR privacy (7), average number of samples after the
down-sampling technique in each party, and average of prediction accuracy (ACC) of 10 trials. We also show averagesof ACC for the centralized and individual analyses.
This result shows that, by the down-sampling technique,we can increase the value of ε-DR privacy (7) without loss of
ACC; see the case of ε = 10−2 . Also, if the predictive accuracy is allowed to decrease slightly, it can take a larger valueof ε-DR privacy (7); see the case of ε = 0.2. These resultsmean that a small number of samples significantly reducethe values of ε-DR privacy (7), while these samples do notsignificantly affect the predictive results. Additionally, evenwith a larger ε, e.g., ε = 0.5, the predictive accuracy (ACC)of the collaborative data analysis is still higher than that of
the individual analysis.

Remarks on privacy analysis
The collaborative data analysis has the following double privacy layer for protection of the private data Xi .

• No one can have the private data Xi because fi is private
under the protocol (Theorems 2 and 3).
• Even if fi is stolen, the private data Xi is still protected regarding ε-DR privacy definitions (5) and (6) (Theorem 4).
For satisfying ε-DR privacy definitions (5) and (6) withcertain quantities ε1 > 0 and ε2 > 0, we need to pay attention to the construction of fi . Dimensionality reduction
method satisfying ε-DR privacy (5) with a given ε1 > 0
has been proposed in (Nguyen et al. 2019). For satisfying
ε-DR privacy (6) with a given ε2 > 0, we can use a downsampling technique, which removes data samples satisfying
dist(x, x0 ) < ε2 , or a constrained dimensionality reduction
method, which adds (6) as a constrain in optimization.
We also observed from our numerical evaluation that the
down-sampling technique can take a larger value of ε-DR
privacy (7) with small decreasing of ACC.
Here, we also shortly discuss the privacy against the membership inference attack. The membership inference attacks
involve constructing multiple reference datasets and observing the change in the output of the constructed model according to the presence or absence of the target data samples. Therefore, they are only feasible in scenarios when individual dimensionality reduction functions fi are leaked in
the collaborative data analysis. To secure data collaborationfrom the collusion of users in applications where membership inference is a concern, specialized dimensionality reduction algorithms providing Differential Privacy, such asin (Jiang et al. 2013), can be also considered.

Conclusions
In this paper, we analyzed the accuracy and privacy of a nonmodel sharing-type federated learning, so-called collaborative data analysis.
From the accuracy analysis, we provided the sufficientcondition (4) for equivalence of the collaborative data analysis and the centralized analysis with dimensionality reduction. We also provided a criteria τ1 for evaluating accuracyof the collaborative data analysis and numerically evaluatedthem.
From the privacy analysis, we proved that, in the collaborative data analysis, the privacy of the private dataset is preserved based on a double secureness against insider and external attacking scenarios. We also evaluated the trade-off
relationship of privacy and accuracy and showed that the
down-sampling technique can take a larger value of ε-DR
privacy (7) with small decreasing of prediction accuracy.
In the future, we will further analyze the accuracy and privacy of the collaborative data analysis for more complicatedsituations, e.g, usage of nonlinear dimensionality reduction
function and the case of vertical and horizontal data distribution, with numerical evaluation in real-world problems.

Acknowledgements
The authors would like to thank the anonymous reviewers
for their constructive comments. This work was supported
in part by the New Energy and Industrial Technology Development Organization (NEDO).

References
Agrawal, R.; and Srikant, R. 2000. Privacy-preserving data
mining. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, 439–450.
Alotaibi, K.; Rayward-Smith, V. J.; Wang, W.; and de la
Iglesia, B. 2012. Non-linear dimensionality reduction for
privacy-preserving data classification. In 2012 International
Conference on Privacy, Security, Risk and Trust and 2012
International Confernece on Social Computing, 694–701.
IEEE.
Dwork, C. 2008. Differential privacy: A survey of results.
In International conference on theory and applications ofmodels of computation, 1–19. Springer.
Fisher, R. A. 1936. The use of multiple measurements intaxonomic problems. Annals of human genetics 7(2): 179–
188.
He, X.; and Niyogi, P. 2004. Locality preserving projections.
In Advances in neural information processing systems, 153–
160.
Imakura, A.; Matsuda, M.; Ye, X.; and Sakurai, T. 2019.
Complex Moment-Based Supervised Eigenmap for Dimensionality Reduction. In Proceedings of the AAAI Conferenceon Artificial Intelligence, volume 33, 3910–3918.
Imakura, A.; and Sakurai, T. 2020. Data Collaboration Analysis Framework Using Centralization of Individual Intermediate Representations for Distributed Data Sets. ASCEASME Journal of Risk and Uncertainty in Engineering Systems, Part A: Civil Engineering 6: 04020018.
Imakura, A.; Ye, X.; and Sakurai, T. 2020. Collaborative
Data Analysis: Non-Model Sharing-Type Machine Learning for Distributed Data. In 2020 Principle and Practice ofData and Knowledge Acquisition Workshop (PKAW2020),(accepted).
Jiang, X.; Ji, Z.; Wang, S.; Mohammed, N.; Cheng, S.; andOhno-Machado, L. 2013. Differential-private data publishing through component analysis. Transactions on data privacy 6(1): 19.
Kairouz, P.; McMahan, H. B.; Avent, B.; Bellet, A.; Bennis,
M.; Bhagoji, A. N.; Bonawitz, K.; Charles, Z.; Cormode, G.;
Cummings, R.; et al. 2019. Advances and open problems infederated learning. arXiv preprint arXiv:1912.04977 .
LeCun, Y. 1998. The MNIST database of handwritten digits.
http://yann. lecun. com/exdb/mnist/ .
Li, Q.; Wen, Z.; and He, B. 2019. Federated learning systems: Vision, hype and reality for data privacy and protection. arXiv preprint arXiv:1907.09693 .
Li, X.; Chen, M.; Nie, F.; and Wang, Q. 2017. Locality Adaptive Discriminant Analysis. In Proceedings ofthe 26th International Joint Conference on Artificial Intelligence, 2201–2207. AAAI Press.
Liu, K.; Kargupta, H.; and Ryan, J. 2005.
Randomprojection-based multiplicative data perturbation for privacy
preserving distributed data mining. IEEE Transactions on
knowledge and Data Engineering 18(1): 92–106.

Maaten, L. v. d.; and Hinton, G. 2008. Visualizing data using
t-SNE. Journal of machine learning research 9: 2579–2605.
McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; andy Arcas, B. A. 2017. Communication-efficient learning ofdeep networks from decentralized data. In Artificial Intelligence and Statistics, 1273–1282.
Mendes, R.; and Vilela, J. P. 2017. Privacy-preserving data
mining: methods, metrics, and applications. IEEE Access 5:10562–10582.
Narayanan, A.; and Shmatikov, V. 2006. How to breakanonymity of the netflix prize dataset. arXiv preprint
cs/0610105 .
Nguyen, H.; Zhuang, D.; Wu, P.-Y.; and Chang, M. 2019.
AutoGAN-based Dimension Reduction for Privacy Preservation. arXiv Preprint arXiv:1902.10799.
Nguyen, H.; Zhuang, D.; Wu, P.-Y.; and Chang, M. 2020.
AutoGAN-based dimension reduction for privacy preservation. Neurocomputing 384: 94–103.
Pearson, K. 1901. LIII. On lines and planes of closest fit
to systems of points in space. The London, Edinburgh,
and Dublin Philosophical Magazine and Journal of Science2(11): 559–572.
Samarati, P.; and Sweeney, L. 1998. Protecting privacy
when disclosing information: k-anonymity and its enforcement through generalization and suppression .
Saunders, C.; Gammerman, A.; and Vovk, V. 1998. Ridge
regression learning algorithm in dual variables .
Sugiyama, M. 2007. Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis.
Journal of machine learning research 8(May): 1027–1061.
Tai, B.-C.; Li, S.-C.; Huang, Y.; Suri, N.; and Wang, P.-C.
2018. Exploring the Relationship Between Dimensionality
Reduction and Private Data Release. In 2018 IEEE 23rd
Pacific Rim International Symposium on Dependable Computing (PRDC), 25–33. IEEE.
Verykios, V. S.; Bertino, E.; Fovino, I. N.; Provenza, L. P.;
Saygin, Y.; and Theodoridis, Y. 2004. State-of-the-art in privacy preserving data mining. ACM Sigmod Record 33(1):50–57.
Zelnik-Manor, L.; and Perona, P. 2005. Self-tuning spectralclustering. In Advances in neural information processing
systems, 1601–1608.
Zhao, J.; Chen, Y.; and Zhang, W. 2019. Differential privacy
preservation in deep learning: Challenges, opportunities andsolutions. IEEE Access 7: 48901–48911.