# Soft Semi-Supervised Deep Learning-Based Clustering
## 4 Experiments
### Results and Discussion
Figure 2: Change trend recorded for the performance metrics obtained using SC-DEC on MNIST (**a**), USPS (**b**), and STL-10 (**c**) datasets. We can notice from Table 7 and Figure 4 that the number of pairwise constraints (\(\beta\) hyper-parameter) was correlated with the dataset size. In the largest dataset, MNIST, the best value was \(\beta=1.5\), compared to \(\beta=0.3\) and \(\beta=0.5\) for the STL-10 and USPS datasets, respectively. However, the change in accuracy in response to the increased \(\beta\) value was not substantial, as the difference between the highest and lowest accuracy equaled 1.05% (MNIST), 0.43% (STL-10), and 0.74% (USPS). This is consistent with the findings in the SDEC paper [38] that the initial introduction of pairwise constraints into deep embedded clustering will lead to a significant increase in performance, and then the performance becomes stable, indicating that enough prior information has been captured. According to SDEC, this observation is generally consistent with the semi-supervised learning literature. Moreover, the effect of the number of constraints \(\beta\) on the performance of the proposed approach may be subject to the pairwise constraints generated, as the generation is random for each new value of the hyper-parameter \(\beta\). This would explain the relative randomness that characterizes the results shown in Figure 4.

Furthermore, Table 7 and Figure 5 show that the highest accuracy values for the fuzzy parameter \(m\) among the three datasets are within the interval suggested by [61] as a heuristic for selecting the m value for FCM-based clustering. The calculation in [61] suggests that the best choice for m is probably in the interval of [1.5, 2.5]. Interestingly, training our model on the STL-10 dataset using a 512-batch size yielded better results than using the default 256-batch size. ## 5 Conclusions and Future Work

Most real-world datasets are weakly labeled by nature, which inspired the utilization of the available prior knowledge as supervision information by clustering algorithms to carry the clustering process away from the local minima. This led to the introduction of semi-supervised learning within the clustering paradigm. However, the state-of-the-art semi-supervised deep clustering approaches remain below expectations compared to the fully unsupervised approaches, with several potentials to be explored. In this paper, we propose a novel semi-supervised deep clustering approach (named Soft Constrained Deep Clustering, SC-DEC) to overcome the limitations in the existing semi-supervised clustering approaches. Specifically, the proposed approach leverages a deep neural network architecture and generates fuzzy membership degrees that better reflect the true partition of the data. Furthermore, the scarcely available prior knowledge was used as side information and formulated as a set of soft pairwise constraints to direct the machine learning process into clustering unlabeled data. This clustering task was formulated as an optimization problem where a novel objective function was minimized to simultaneously discover the hidden data clusters and optimize the deep neural network. The proposed approach was assessed using standard datasets and performance measures. The experiments proved that the proposed approach can automatically learn the partitioning of data. Moreover, various calibrations of the model's hyper-parameters were investigated during the experiments. The experimental results showed that the size of the pairwise constraints was positively correlated with the dataset size. However, after enough prior information has been captured via the initial insertion of the pairwise constraints into the proposed model,

Figure 7: Effect of data batch size on SC-DEC clustering accuracy for MNIST (**a**), USPS (**b**), and STL-10 (**c**) datasets. the model performance became stable. Furthermore, when compared to the state-of-the-art approaches, SC-DEC produced competitive results on the STL-10 dataset and outperformed the other models on MNIST and USPS.