# Dynamic Depth Learning in Stacked AutoEncoders
## 1 Introduction  
Findings in the natural sciences along with recent technological advances have promoted the mimicking of the human nervous system/network. In particular, this has led to the flourishing of machine learning (ML) as a subfield of the artificial intelligence domain [1]. Specifically, ML has gained the attention of researchers who aim to design and introduce intelligent solutions for computing applications that span a very large spectrum of domains. Mainly, such exceptional interest is motivated by the availability of a huge amount of data. Moreover, drastic improvements in computing capabilities and resources have further boosted researchers' efforts to present promising ML techniques. Consequently, ML has been deployed in various domains and it has found application in various research fields such as speech and handwriting recognition [2, 3], natural language processing [4], computer vision [5], image processing [6], banking and credit card fraud detection [7], etc. The advancements in artificial neural network (ANN)-related research laid the cornerstone of deep learning (DL) [8]. In fact, the term \"deep\" refers to the concept of having the data transformation deployed through numerous layers [1]. Another important characteristic of deep learning is automatic feature extraction which alleviates the problem of feature handcrafting and engineering [8]. Given the success that it has achieved recently in various domains, DL has become a universal learning approach for many research communities [8]. Actually, it has proved to be effective in many fields. In particular, it yields outstanding results in image processing, computer vision, natural language processing, object detection, face recognition, and handwriting and speech recognition [1]. Furthermore, there are successful commercial applications using DL that have been established by information technology giants such as Google [9], Apple [10], Facebook [11], and Amazon [12], who use deep learning techniques in their applications. Deep learning has been adapted for both supervised and unsupervised learning applications. AutoEncoders (AE) have been introduced as a main DL architecture designedto address unsupervised learning problems [1]. AE is a neural network composed of an input layer, a hidden layer, and an output layer. Through its hidden layer, the network learns the code which allows reproduction of the input. Such AEs can be stacked by associating successive hidden layers. This results in a deep learning topology referred to as the stacked AutoEncoder [13]. AutoEncoders are typically used for embedding learning and low-dimensional representation [13]. In addition to the dimensionality reduction and compression applications, AEs have been used for anomaly detection, image denoising, image generation, and feature extraction [13]. They have been also employed in several fields such as bioinformatics [14], cyber-security [15], recommender systems [16], etc. Furthermore, they have been integrated into other complex supervised deep learning paradigms as encoding and decoding modules [8]. Despite researchers' achievements in designing deep-learning-based solutions for various applications, some challenges still face existing DL architectures. In particular, they suffer from the interpretability limitation. Actually, DL approaches are considered black boxes that are unable to provide a logical justification of the obtained results. Another issue consists in the large number of hyperparameters to be selected and fine-tuned during the training phase. In particular, the number of hidden layers, and the number of nodes at each layer which represent the network depth and the network width, respectively, affect the model performance. In fact, overly deep and/or wide networks yield problems such as vanishing gradient [8], long training time [13], and overfitting [13]. Therefore, setting the appropriate depth and width of the network is perceived as a cornerstone in the design of an effective deep learning topology.