# Soft Semi-Supervised Deep Learning-Based Clustering
## 3 Proposed Soft Constrained Deep Clustering (SC-DEC)  
The proposed semi-supervised deep clustering approach, named Soft Constrained Deep Clustering (SC-DEC), leverages a deep neural network architecture for feature learning and performs clustering with fuzzy membership degrees. Specifically, the deep clustering problem is formulated as an optimization of a novel objective function that is designed to simultaneously discover the hidden data clusters and optimize the deep neural network. Furthermore, the available supervision knowledge is incorporated as soft pairwise constraints within the objective function in a soft way such that the compliance to a constraint is not strictly obligated. Moreover, data partition is represented in a fuzzy manner within the objective to accurately reflect the grouping of data. This section introduces the design details of the proposed SC-DEC approach. The block diagram of SC-DEC is depicted in Figure 1. The proposed semi-supervised deep embedded clustering approach relies on two main components: (i) an objective function that carries out the clustering tasks as proposed, and (ii) an autoencoder AE network that learns the discriminative embedded representation of the original data. Let \(X{=}\left\{x_{i}\right\}_{i=1}^{n}\) be a set of unlabeled data that serves as the input to our proposed clustering method, where each sample \(x_{i}\in\mathbb{R}_{d}\) and d is the dimensions of the input space. Instead of performing the clustering task directly in the original data space, we define a nonlinear mapping function \(f_{\theta}\) to transform the original input data \(X\) into a latent feature space \(Z\); \(f_{\theta}:X\to Z\), where \(\theta\) represents the learnable parameters. The dimensionality of the latent space \(Z\) is much lower than the original space dimensionality. The main goal of the proposed SC-DEC method is to output an appropriate clustering of data in the embedded feature space \(Z\) by utilizing the unlabeled data and the injected supervision. Specifically, it aims to partition the data input into \(k\) clusters, where each cluster is defined by a centroid \(\mu_{j=1,...,k}\) and \(\mu_{j}{\in}\mathbb{R}_{d}\). Specifically, for k-means algorithms, the number of clusters \(k\) is based on the related literature and the considered benchmark datasets. Basically, we aim to find a cluster-friendly \(f_{\theta}\) such that the learned parameters are biased towards the clustering task and the available knowledge. To initialize the parameters of the \(f_{\theta}\), a deep autoencoder network [54] is built and trained in an unsupervised manner. Later, only the encoder layers are used within our model to receive the input data and transform it to the embedded space. The supervision knowledge are defined in the proposed approach as a set of should-link soft constraints and denoted by \(\mathbb{C}=\left\{(x_{i},x_{k}):x_{i}\text{ and }x_{k}\text{ should be assigned to the same cluster, }1\leq i,j\leq N\right\}\). These soft constraint sets are pre-defined and randomly generated from the dataset and generally, the supervision information should be available for a subset of the dataset only. In the Experiments section, we vary its ratio and investigate the performance. We introduce our algorithm as an improvement to deep embedded clustering algorithms. Specifically, we propose to integrate should-link constraints into the clustering loss objective of DEC [21] to find clustering-friendly representations. Consequently, we tackle the proposed optimization problem by defining the following objective function  
Figure 1: Overview of the proposed approach (SC-DEC).