# Double Deep Autoencoder for Heterogeneous Distributed Clustering
###### Abstract  
Given the issues relating to big data and privacy-preserving challenges, distributed data mining (DDM) has received much attention recently. Here, we focus on the clustering problem of distributed environments. Several distributed clustering algorithms have been proposed to solve this problem, however, previous studies have mainly considered homogeneous data. In this paper, we develop a double deep autoencoder structure for clustering in distributed and heterogeneous datasets. Three datasets are used to demonstrate the proposed algorithms, and show their usefulness according to the consistent accuracy index.  
distributed data mining (DDM); clustering; big data; heterogeneous databases; deep autoencoder +
Footnote †: journal: Information Information  
extension of parallel computing. On the other hand, [8] first proposed the collective hierarchical clustering (CHC) algorithm to consider the distributed clustering problem in heterogeneous databases. Later on, several papers on heterogeneous distributed clustering algorithms, such as those by [9; 10; 11], were proposed to consider more complex situations [12].  
Usually, heterogeneous databases can use principle component analysis (PCA) and correlation analysis to reduce the number of dimensions and consider the interactive relations between features in past research [13]. However, as PCA and correlation analysis do not consider nonlinear representative or nonlinear interactive relations between features, they might lose some important information for clustering. Hence, in this paper, instead of using PCA, we consider the double deep autoencoder model for the above issues, as follows: the stage of the replica neural network learns the nonlinear codes from each local dataset. Then, we concatenate all local codes to be the input of the server neural network, to learn the nonlinear global codes across the replicas; that is, the interactive relationships between the features. We use the double deep autoencoder and distinguish the proposed method from others.  
In this paper, we define our problem as follows. Assume \(k\) parties, \(P_{1},\ldots,P_{k}\), own large private datasets, \(D_{1},\ldots,D_{k}\). They want to apply a clustering algorithm to the whole dataset without simply joining all datasets, due to a big data issue or privacy-preserving reason. Hence, the information that a party can use is the model output of other parties. Note that no trusted party is considered here. In addition, we consider a heterogeneous view of the datasets, which is that all parties hold exclusive information about the features of all the records. To overcome this problem, we developed a double deep autoencoder to extract the nonlinear important features by considering the information from the self and other parties. Compared with past papers, the original contribution of this paper is the integration of the deep autoencoders, and clustering with the concept of deep learning. Three heterogeneous distributed datasets are used to demonstrate the proposed algorithms and the ability to overcome our problem. Therefore, the contribution of this paper is the proposal of a new deep learning model, specifically the double deep autoencoder, to deal with distributed and homogeneous datasets.  
The rest of this paper is organized as follows: the distributed clustering algorithm is introduced in Section 2. The proposed double deep autoencoder used in the distributed environment is presented in Section 3. Experiments are given in Section 4, and the last section presents the discussion and conclusion.