# A Review of Deep Learning Algorithms and Their Applications in Healthcare
## 3 Artificial Neural Networks
### Autoencoders (AE)
#### 3.2.2 Types of Autoencoders  
1. Convolution Autoencoders:  
In convolutional autoencoders, the input signals are encoded in simple signals to reconstruct the image or modify its reflectance as shown in Figure 4.  
The applications of convolutional encoders are image reconstruction, image colorization, latent space clustering, and generating higher resolution images [46]. 2. Sparse Autoencoders (SAEs):  
By using sparse autoencoders, an information bottleneck is introduced without reducing the number of nodes in our hidden layers as shown in Figure 5. Rather, the loss function is to be constructed to penalize activations within a layer. Feature extraction from raw data is the core purpose of SAEs. The sparsity of the object presentation can be achieved using two methods. One method is to penalize the hidden unit's bias, and the other is to directly penalize the hidden unit's activation output [46; 47]. Figure 4: A representation of a convolutional autoencoder [46]. Figure 3: A representation of an autoencoder [46]. 3. Deep Autoencoders:  
A deep autoencoder is an extension of the simple autoencoder. Deep autoencoding is implemented in layers, with the first layer being used for first-order features. Second-order features correspond to patterns in the appearance of first-order features in the second layer. Higher-order features can be learned by deeper layers of the deep autoencoder. A deep autoencoder is composed of two symmetrical deep belief networks as shown in Figure 6. The first four or five shallow layers represent the encoding part, and the second set of four or five layers makes up the decoding part [46]. The applications of deep autoencoders are image search, data compression, topic modeling, and information retrieval. 4. Contractive Autoencoders:  
The contractive autoencoder is an unsupervised deep learning technique that enables neural networks to encode unlabeled training data. This is accomplished by constructing a loss term that penalizes large derivatives of hidden layer activations with respect to input  
Figure 5: A representation of a sparse autoencoder [46]. Figure 6: A representation of deep autoencoder [46]. training examples, essentially penalizing instances: a small change in the input leads to a large change in the encoding space [46; 48]. * Denoising Autoencoders:  
This is a type of autoencoders that through modifying the criterion of reconstruction can try to enhance the representation that retrieves useful features. Therefore, it decreases the danger in learning an identity function [49]. It accepts a data point that is corrupted; then training is used to obtain the original input that is undistorted as the output, and this by diminishing the average reconstruction error in training data--for instance, denoising, or the corrupted data are cleaned. A denoising autoencoder can be applied for automatic pre-processing; e.g., it can pre-process an image automatically, thereby enhancing the quality for accuracy of recognition. Stacking denoising autoencoders to initialize a deep network works in a similar fashion as stacking RBMs in deep belief networks or ordinary autoencoders. The input corruption is only used in the initial denoising training of each layer so that it can learn useful feature extractors. After the mapping has been learned, it will be used on uncorrupted inputs from then on. In particular, no corruption is applied to produce the representation that will be used as the input for the next layer of training. The highest-level output representation of a stack of encoders can then be used as input to a stand-alone supervised learning algorithm, for example a support vector machine classifier or (multi-class) logistic regression. Another option is to add a logistic regression layer on top of the encoders, resulting in a deep neural network suitable for supervised learning. Gradient-based procedures such as stochastic gradient descent can be used to fine-tune the parameters of all layers simultaneously [49].