# Dynamic Depth Learning in Stacked AutoEncoders
## 5 Experiments
### Experiment 4: Performance Comparison of DDSAE with State-of-the-Arts  
In this experiment, we aim to compare the performance of the DDSAE algorithm to the state-of-the-art approaches which learn the depth of an SAE in an unsupervised manner. The considered approaches are the related works published in [26, 27]. The work in [26], referred to as \"Unsupervised restricted depth and width learning for a multi-layer AE\", learns the width and depth of VAE using evolutionary search techniques. It is designed in such a way that the depth cannot exceed five layers and the width varies from 50 to 1000 nodes by a step of 50. The optimization of the depth or the width is achieved using the mutation operator and the fitness is defined as the inverse of the VAE reconstruction error. Similarly, the work in [27], called \"Unsupervised width and depth learning using a chromosome of fixed length\", is a genetic model which optimizes AE width and depth. Specifically, it employs crossover and mutation genetic operators on a 14-gene chromosome to create new candidate solutions. Particularly, the genes of the chromosome encode the number of layers (depth), the number of nodes per layer (width), the activation function, and the loss function. In the following, we refer to the works in [26, 27] as \"Approach1\" and \"Approach2\", respectively. The four datasets previously considered (MNIST [35], CIFAR-10 [36], Parkinson's [37, 38], and Breast Cancer [39, 40]) are used to assess the performance of the DDSAE approach and compare it to the state-of-the-art approaches. Moreover, the SoftMax classifier was appended on top of the encoder part after training the AE. Tables 6-9 depict the performance of DDSAE and compare it with the two state-of-the-art approaches on the four datasets along with the learned depth. As can be seen in Table 6, on the MNIST dataset [35] DDSAE achieved 97% with respect to the considered performance measures. Hence, it outperformed the first approach [26] by \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Approach** & **Accuracy** & **F1-Score** & **Recall** & **Precision** & **Depth** \\ \hline DDSAE & **97\%** & **97\%** & **97\%** & **97\%** & \(1\) \\ Approach 1 [26] & 75.98\% & 74.38\% & 75.16\% & 78.4\% & \(1\) \\ Approach 2 [27] & 91.26\% & 91.14\% & 91.13\% & 91.2\% & \(1\) \\ \hline \hline \end{tabular} Bold numbers indicate best performance.  
\end{table} 
Table 6: Performance Comparison of DDSAE with state-of-the-art on MNIST. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Approach** & **Accuracy** & **F1-Score** & **Recall** & **Precision** & **Depth** \\ \hline DDSAE & **38\%** & **36.5\%** & **38\%** & **38.6\%** & \(1\) \\ Approach 1 [26] & 20.99\% & 17.22\% & 20.99\% & 17.15\% & \(1\) \\ Approach 2 [27] & 32.02\% & 29.7\% & 32.02\% & 35.63\% & \(1\) \\ \hline \hline \end{tabular} Bold numbers indicate best performance.  
\end{table} 
Table 7: Performance Comparison of DDSAE with state-of-the-art on CIFAR-10. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Approach** & **Accuracy** & **F1-Score** & **Recall** & **Precision** & **Depth** \\ \hline DDSAE & 68.57\% & **68.1\%** & 68.1\% & 68.3\% & \(1\) \\ Approach 1 [26] & 48.57\% & 40.34\% & 45.72\% & 42.52\% & \(4\) \\ Approach 2 [27] & 54.29\% & 35.18\% & 50\% & 27.14\% & \(1\) \\ \hline \hline \end{tabular} Bold numbers indicate best performance.  
\end{table} 
Table 9: Performance Comparison of DDSAE with state-of-the-art on Breast Cancer. depth of an SAE in an unsupervised manner in terms of classification accuracy in balanced datasets, and in terms of F1-score in imbalanced datasets. Moreover, DDSAE outperformed the state-of-the-art approaches in the resulting architecture on the imbalanced datasets. The accuracy, F1-score, precision, and recall achieved on Parkinson's and Breast Cancer are displayed in Figures 15 and 16, respectively. Real-world data exhibit generally overlapping and nonlinearly separable inter-class boundaries.