# Soft Semi-Supervised Deep Learning-Based Clustering
## 2 Related Works
Specifically, the IDEC model [37] extended DEC by preserving the local structure of data in the feature space through keeping the decoder layers to avoid feature space distortion by the clustering loss. Another model introduced in [27] extended IDEC by adopting a convolutional autoencoder (CAE) as the deep network. Later, several deep clustering works based on (CAE) were suggested [32]. Data augmentation was also associated with DEC in [28] to improve the model's generalization by making the DNN learn more representative features. Moreover, a k-means based DEC framework was presented in [19]. In particular, a cost function that consists of dimensionality reduction, data reconstruction, and a k-means clustering structure promoting regularization loss terms was introduced. Another related DEC-based work [33] leverages symmetry-based distances within the DEC framework as a powerful tool to distinguish symmetric shapes. The authors in [38] observed the fact that DEC [21] does not make use of prior knowledge to guide the learning process. They extended DEC [21] and proposed a new scheme of Semi-Supervised Deep Embedded Clustering (SDEC) to overcome this limitation. SDEC's best performance was reported as an 86.11% clustering accuracy and 82.89% NMI on the MNIST image dataset. Since then, Semi-Supervised Deep Embedded Clustering (SS-DEC) became a growing field. Actually, most of the SS-DEC literature addresses general clustering purposes [39; 40; 41; 42]. On the other hand, some works targeted anomaly detection [43], software fault proneness classification (fault prediction) [44], image classification and segmentation [45], and deep generative purposes [46; 47]. The supervision injected in SS-DEC models takes several forms. The researchers in [41] imposed standard must-link and cannot-link pairwise constraints to their model, while the authors in [31] extended the encoding of standard pairwise constraints to more complex constraints such as continuous values (triplet constraints), instance difficulty constraints, and cluster-level balancing constraints. Triplet constraints are useful in the cases where no strong pairwise guidance is available. As for the instance difficulty constraints, it allows the user to a priori specify which instances are easier to cluster (i.e., they belong strongly to only one cluster), whereas the cluster-level balancing constraints enable the experts to guide the clustering process via the prior cardinality information. Another approach, Ts2 DEC, outlined in [48], used triplet constraints. In [49], the pairwise constraints are self-generating from a mutual KNN graph which makes the clustering approach unsupervised. The method then extends it to semi-supervised clustering by including human intervention to finetune these self-generating constraints by analyzing the losses associated with the pairs to form a set of false positive candidates. ClusterNet [50] also uses pairwise semantic constraints from very few labeled data samples (<5% of total data) and exploits the abundant unlabeled data to drive the clustering. The network is optimized by minimizing a combination of k-means-based clustering loss and pairwise KL-divergence loss where the two are regularized via an autoencoder's reconstruction loss and each are defined for both the labeled as well as unlabeled data. A different approach to include supervision information was outlined in [47] where a small, labeled dataset is used to assign classes to components of Gaussian mixtures. The resulting mixture describes the distribution of the whole data. The authors in [39] extended deep embedded clustering approaches to Electronic Health Record (I) patient cohorts. Specifically, supervision was applied by modifying the latent representation according to known patient subgroups through applying transfer learning of the encoder and fine-tuning of layers. One should note that most DEC-based works [38; 39; 45; 48] use k-means clustering algorithms to initialize the cluster centers, followed by cluster assignment hardening in the form of clustering loss.