# Soft Semi-Supervised Deep Learning-Based Clustering
## 4 Experiments
### Experimental Scenarios  
After conducting preliminary investigations and experiments using the proposed approach, it was observed that higher numbers of training epochs can result in a poor clustering performance. Moreover, the clustering results are sensitive to the initialization. Furthermore, the Adam optimizer outperformed the other optimizers, with a significant accuracy advantage. Accordingly, a lower range of training epochs was set for all datasets. Moreover, random seeds were fixed for all implementation libraries. Specifically, the seeds values were set based on the performances recorded in three trial runs. During the preliminary experimentation on model optimizers, the hyper-parameters used for all datasets were set as follows: batch size = 256, \(\gamma\) and \(\beta\) = 1, and \(m\) = 1.2 (hyper-parameters are defined in Section 3). As for the number of pretraining epochs, 1000 was used for USPS, 2000 for MNIST, and 20 for STL-10. Table 2 shows the accuracy values resulting from using the different model optimizers and optimizers properties. It is \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset Name and** **Reference** & **Number of Points** & **Number of Classes** & **Number of Dimensions** \\ \hline MNIST [57] & 70,000 & 10 & 784 \\ \hline USPS [58] & 9298 & 10 & 256 \\ \hline STL-10 [59] & 13,000 & 10 & 1428 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Details of the datasets used in this research. noticeable from Table 2 that the Adam optimizer outperformed the other optimizers for all datasets and with a large margin in the STL-10 dataset. After setting Adam as the model optimizer for our proposed approach, we performed other preliminary experiments to investigate the proper range for tuning the number of AE pretraining epochs. Table 3 shows the settings adopted to cluster the datasets using the proposed approach along with the Adam optimizer. Regarding the settings in Table 3, some of the experiments had specific settings. For USPS, in the 50-epoch case, \(\gamma=-100\), \(\beta=0.5\), and \(m=1.5\). In the 100-epoch case, \(\gamma=1\) and \(m=2\). For the MNIST dataset, \(m=1.5\) in the 300-epoch case and \(m=2\) in the 400-epoch case. Table 4 shows the preliminary experiment results in terms of the accuracy values over the number of AE pretraining epochs. For the STL-10 dataset, a smaller number of epochs led to better clustering accuracy. Moreover, for all datasets, a higher number of pretraining epochs was not correlated with the increasing clustering accuracy. Based on that, we selected the range for tuning the number of AE pretraining epochs to be within smaller values that are relative to the size of the dataset. It should also be mentioned that as the Adam optimizer was used, the tuning scenario dedicated for the setting of the (optimizer) hyper-parameter was excluded. For all experiments conducted using the different datasets, the relevant hyper-parameters were tuned for a better initialization and setting, as shown in Table 5.  
In Table 5, since it is the first parameter to be tuned, the number of pretraining epochs does not require default value initialization. As for the batch size's default value, it was initialized according to the value that is widely used by the clustering research community. Regarding \(\beta\) and \(\gamma\) default values, they were initialized with neutral values. As for the initialization of the Fuzzifier m default value, the most frequently used and accepted value in various applications is \(m=2\)[61]. However, when datasets have significant uneven distributions in the cluster sizes, a smaller fuzzifier value has been suggested for the FCM-based clustering algorithms [62].