# Improved Selective Deep-Learning-Based Clustering Ensemble
## 2 Related Work
### Deep Clustering  
The emergence and development of deep clustering methods provides a technique that exploits the commonalities of unsupervised deep networks and clustering to simultaneously learn feature representations and cluster assignments. Deep clustering typically involves learning a mapping from the original data space to a low-dimensional feature space and iteratively optimizing the clustering objective in that feature space [14]. DEC [27] was one of the earliest deep clustering methods. It was inspired by deep learning for computer vision and extended it to unsupervised data clustering. It has been cited in more than 2000 articles and has attracted much attention in the field. DEC uses a pre-trained stacked denoising autoencoder to learn deep feature representations, and then defines a probability distribution based on the centroid and minimizes its KL divergence to an auxiliary target distribution in order to improve the clustering assignment and the feature representations at the same time. IDEC [28] was an improvement on DEC, and argues that the clustering loss defined by DEC corrupts the feature space and leads to unrepresentative features. Therefore, it adds back the decoder to optimize the reconstruction error and clustering loss. Clustering features suitable for local structure preservation are learned by adding the denoising autoencoder of the decoder. DCEC [29] is the convolutional autoencoders version of IDEC, and takes advantage of convolutional neural networks to better extract the hierarchical embedding features. It is superior to stacked autoencoders by incorporating spatial relationships between pixels in images. DFKM [30] is a novel deep clustering method with adaptive loss function and entropy regularization that employs fuzzy k-means and uses fuzzy information to represent a clear deep clustering structure. To enhance the robustness of the model, DFKM utilizes a robust loss function with adaptive weights. GFDC [31] utilizes a deep convolutional network for the feature generator and a graph convolutional network with a softmax layer to perform a clustering assignment. It constructs a topological graph to express the spatial relationship of features. These AE-based methods have the advantage of being easy to implement and extend, but are sensitive to network initialization parameters and hyperparameters, and they also require a certain limited number of layers in the deep network structure. GMVAE [32] assumes that observations are generated from a multi-modal prior distribution and constructs an inference model that can be directly optimized using reparameterization techniques. VaDE [33] embeds the probabilistic clustering problem into a variational autoencoder framework. It models the data generation process through a GMM model and a neural network. VaDE is optimized by maximizing an evidence lower bound on the log-likelihood of the data using a stochastic gradient variational Bayesian estimator and a reparameterization technique. The main difference between GMVAE and VaDE is that the assumed sample-generating model is different and GMVAE is somewhat more complex than VaDE, with poorer empirical results. DCVA [34] considers that the potential space of the autoencoder does not pursue the same clustering goals as the k-means or Gaussian mixture model. Therefore, it introduces a variational autoencoder probabilistic approach that represents distances in the latent space in terms of KL divergence and uses probability distributions as inputs instead of using points in the latent space. Finally, the potential space is clustered using a Bayesian Gaussian mixture model. These VAE-based methods have the advantage of being able to generate samples with reasonable theoretical guarantees, but have the disadvantage of high computational complexity. Witnessing the great success of generative adversarial networks (GANs) in estimating complex data distributions, DAC [35] introduces generative adversarial networks [36] to deep clustering.