# A Hybrid Autoencoder Network for Unsupervised Image Clustering
## 3 Hybrid Autoencoder Network for Image Clustering  
The major characteristic of the CAE is its ability to learn relevant features from an image. It has been successfully used in identifying or recognizing images. The CAE inherits the characteristics of the CNN and attains better performance than traditional methods in image clustering. On the other hand, AAE provides another perspective to extract the feature representation from adversariallearning. In this paper, we integrate three kinds of autoencoder networks as follows: First, we consider training convolutional and adversarial autoencoder networks to find the feature representation; then, we concatenate the two feature representatives to form another deep autoencoder network to obtain the high-level feature representation for clustering images, as shown in Figure 2.  
In this paper, the proposed method has three objectives. For the CAE process, we set our image (\(x\)) as the input layer and add multiple convolutional layers to train the encoder. The encoder output given an input image is denoted by \(q(z_{1}|x)\). Then, we can derive the representation code, also called the latent vector. Finally, the decoder output (\(p(x|z_{1})\)) will derive the reconstructed image (\(x^{\prime}\)). Let the encoder \(h=F_{\omega}(x_{i})\) and decoder \(x^{\prime}=G_{\omega^{\prime}}(h)\). The loss function of CAE is used to minimize the reconstruction error as follows:  
\[L_{\alpha e}=\frac{1}{n}\sum_{i=1}^{n}\|G_{\omega^{\prime}}(F_{\omega}(x_{i}) )-x_{i}\|_{2}^{2} \tag{4}\]  
where \(n\) is the number of images in the dataset, and \(x_{i}\in\mathbb{R}^{2}\) is the \(i\)th image. The characteristic of AAE is that it is a probabilistic AE used to perform variational inference by matching the aggregated posterior of the coder of the autoencoder with an arbitrary prior distribution [21]. The content of AAE is similar to VAE from the perspective of matching the distribution of the latter vector. However, VAE uses Kullback-Leibler divergence and AAE employs adversarial training, which means that we only need to sample from prior distribution to induce the  
Figure 2: Structure of the hybrid autoencoder. CAE: convolutional autoencoder; SAE: stacked autoencoder; and AAE: adversarial autoencoder. coder, rather than having access to the exact functional form of the prior distribution. The success of AAE in image clustering has also been reported by [PERSON] et al., 2018 and [PERSON] et al., 2016. For the AAE process, its main difference from CAE is that AAE has another discrimination step to distinguish between the true and mapped target embeddings. The purpose of the encoder and the decoder is to try to fool the discriminator and obtain good reconstructions. The training of an AAE has two phases: The first is the reconstruction phase, which updates the encoder and decoder to minimize reconstruction error, and the second is the regularization phases, which update the discriminator to distinguish true prior samples from generated samples. Hence, the loss function of AAE is represented as  
\[L_{\text{ane}}=E_{x}\Big{[}E_{q(\pm i)}(-\log p(x|z))\Big{]}+E_{x}[KL(q(z|x) \big{\|}p(z))] \tag{5}\]  
After obtaining the feature representations from CAE and AAE, we concatenate these co-representation codes to form another SAE and predict the clusters of images by the representation codes. The code layer is a compressed feature vector used to obtain the high-level information of input features and reduce the dimensions of the features dramatically. The code layer is useful information for reducing the dimensionality of input features and increasing the accuracy of the models [24].