# Review of Vibration-Based Structural Health Monitoring Using Deep Learning
## 4 Brief Introduction to Deep Learning and Its Future Applications in Vibration Analysis  
This section introduces categories of deep learning that can be applied to many areas of vibration-based monitoring. [PERSON] and [PERSON] [78] defined four major architectures:  
* Unsupervised pretrained networks (UPNs);
* Convolutional neural networks (CNNs);
* Recurrent neural networks;
* Recursive neural networks (RNNs). A UPN gives a computer unlabeled data for training. The computer learns and organizes the data by itself to teach patterns in the data to the user. The UPN works with unsupervised pretrained input data to get the weights for the first layer. The performance of this training progressively improves to the output layer of the whole network, which is sorted by experience. Typical architectures for UPNs include the autoencoder, DBN and GAN. An autoencoder reduces the dimensions of a dataset and reconstructs the output data form. A DBN is made up of multiple restricted Boltzmann machine (RBM) layers that extract higher-level features of the input vector. While an autoencoder uses backpropagation for weights, an RBM learns stochastically. A GAN is composed of two networks: the discriminator and the generator. The generator creates results from the input data and lets the discriminator distinguish if it is real or created. Over multiple iterations, both networks evolve together. In machine learning research, many acronyms are used that give information on the layers. The names and characteristics of examples of UPN applications are given here. [PERSON] [79] proposed a sparse autoencoder (SAE) that uses more nodes for the hidden layers than the input layer. [PERSON] et al. [80] proposed a denoising autoencoder (DAE) that puts some noise in the input layer to prevent unlearning during training. Later, [PERSON] et al. [81] layered multiple DAEs to develop the stacked DAE (SDAE). [PERSON] et al. [82] proposed the contractive autoencoder (CAE), which is robust against small variations and outperforms the DAE at feature extraction. [PERSON] and [PERSON] [83] proposed a generative model of autoencoders called the variational autoencoder (VAE). Many architectures have been proposed to improve upon early models of the GAN. [PERSON] et al. [84] proposed a deep convolutional generative adversarial network (DCGAN) to stabilize the training ability of GAN with a convolution layer. [PERSON] et al. [85] proposed a least-squares GAN (LSGAN) that uses the least-squares loss instead of the sigmoid cross-entropy loss in the discriminator to overcome the vanishing gradient. [PERSON] et al. [86] replaced the discriminator network with an autoencoder, which they called an energy-based GAN (EBGAN). [PERSON] et al. [87] adopted the Wasserstein distance for the loss function to solve mode collapse. CNNs are widely used in image recognition. A representative CNN is composed of convolution, pooling, and fully connected layers. In the convolution layer, features are obtained from a small part of the data. The pooling layer reduces the number of parameters for a large image. The fully connected layer computes the class score as the output. There are many variations of the CNN architecture. [PERSON] et al. [88] proposed the Lenet classic CNN architecture. [PERSON] et al. [89] successfully applied a CNN for the first time in computer vision with AlexNet. Many attempts have been made to stack deep layers. [PERSON] and [PERSON] [90] showed the importance of depth for a CNN called VGGNet with Nineteen layers and [PERSON] et al. [91] used a twenty-two layer CNN called GoogLeNet with many variation of filters in one layer. As the number of layer increase, gradient vanishing occurs. [PERSON] et al. [92] used a residual block to solve gradient vanishing in ResNet, which has over 1000 layers. [PERSON] et al. [93] proposed DenseNet; ResNet combines features through summation, while DenseNet combine features by concatenation. The main concept of a recurrent neural network is the consideration of time steps.