# Soft Semi-Supervised Deep Learning-Based Clustering
## 1 Introduction
Moreover, the researchers' interest in deep clustering paradigms was intended to address challenges such as the inability to handle datasets that lie on nonlinear manifolds, the curse of dimensionality, and the sensitivity to noise [17; 18]. Basically, the earliest deep clustering works [19; 20] focused on feature transformation and clustering as two independent processes. In other words, the data were first mapped into a new feature space and then fed into a clustering algorithm. Recently, deep clustering has been adapted to jointly perform feature learning, transformation, and the clustering of data that exhibit a highly complex latent structure [5]. The fundamental component of deep clustering approaches relies on deep neural network architectures such as autoencoders [21], the network loss, and the clustering loss optimization [7]. Deep clustering potentials were mainly explored in the context of entirely unsupervised learning. However, the datasets are naturally weak or poorly labeled in the real world [22]. This boosted the researchers' efforts to exploit this knowledge to inject some supervision to guide the clustering process, i.e., introduce the semi-supervised learning paradigm. Semi-supervised learning uses both labeled and unlabeled data to train a model. Moreover, semi-supervised learning exploits prior knowledge and formulates it as constraints to ease the learning process. Despite researchers' efforts, the state-of-the-art, semi-supervised, deep clustering approaches remain far below expectations compared to the fully unsupervised approaches. In particular, the use of fuzzy logic was not investigated to represent the data partition in the context of semi-supervised deep learning. Moreover, the existing semi-supervised clustering methods only proposed the integration of the supervision information as crisp pairwise constraints rather than soft ones. In this research, we propose a novel semi-supervised deep clustering approach, named Soft Constrained Deep Clustering (SC-DEC), to overcome the limitations in current approaches. The proposed approach leverages a deep neural network architecture for feature learning and performs clustering with fuzzy membership degrees to better represent the true partition of the data. Specifically, we formulated the deep clustering problem as an optimization of a novel objective function. This function is designed to simultaneously discover the hidden data clusters and optimize the deep neural network. Moreover, soft pairwise constraints were incorporated within the objective function to represent the available supervision knowledge. These constraints were formulated in a relaxing way in which the compliance to a constraint is not strictly obligated, which makes the proposed approach more suitable for real-world data clustering applications where the available side information is not mature enough to be strictly imposed. Additionally, the fuzzy-based representation of data partition was adopted in the proposed method to reflect the grouping of data in a more accurate way. The main contributions of this research can be summarized as follows: (i) We formulated the proposed semi-supervised deep clustering approach as an optimization problem and designed the objective functions that carry the proposed tasks to simultaneously learn a discriminative embedded representation of the original data along with the optimal data clusters. (ii) We solved the formulated optimization problem and derived updated equations of the respective parameters. (iii) We designed and implemented a novel deep semi-supervised clustering algorithm using state-of-the-art technology, platforms, and tools. (iv) We evaluated the performance of the proposed approach using real datasets and standard performance measures in addition to an objective comparison with relevant state-of-the-art approaches. The rest of this paper is organized as follows: Section 2 presents the related works while Section 3 introduces the proposed method.