# A Novel Parallel Auto-Encoder Framework for Multi-Scale Data in Civil Structural Health Monitoring
## 1 Introduction
Since various layers can abstract various structures of the data, it is possible to learn the true underlying structure and the non-reducible nonlinearities of the original data. The typical training process for DLNN consists of unsupervised feature learning that facilitates the process of extracting features implicitly in comparison to other feature learning methods that have explicit formulations. DLNN is a popular technique in machine learning community with applications ranging from computer vision [16; 17], natural language processing [18; 19] and audio processing [20] and so forth. It has recently been applied to the community for SHM via big data analysis [21; 22]. A vision-based method using a deep architecture of Convolutional Neural Networks (CNN) was proposed in [PERSON] et al. [22] for detecting concrete cracks without calculating the defect features. Later, they have developed a Faster Region-based Convolutional Neural Networks (Faster R-CNN) [23] method to detect multiple types of surface damages, which has better computation efficiency in comparison with CNN-based method of [PERSON] et al. [22]. Moreover, the Faster R-CNN has been used with UAVs for damage detection recently in 2018 [24]. Auto-encoder is one kind of unsupervised learning method that has usually been used to produce a lower-dimensional representation for data, which preserves the information that is useful for the ultimate task. By stacking multiple hidden layers of auto-encoders, a Deep Auto-encoder (DAE) model can be established to learn high-level features with an unsupervised learning algorithm. Particularly, DAE can be utilised for performing non-linear regression task [25; 26] with a supervised learning algorithm. Recent progress in Reference [3] shows a promising future for deep auto-encoder model (named AutoDNet) in comparison with traditional ANNs with applications in SHM. They have demonstrated that AutoDNet can achieve good performance for structural damage detection/identification via two stages: effective dimensionality reduction and accurate relationship learning [3]. A layer-wise pre-training scheme is performed in both two stages. After pre-training, the obtained weightings should be close to a global optimal solution. Then, the entire network is fine-tuned with respect to ultimate objective function. Also, the hyperbolic tangent function named 'tanh' has been utilized as the activation function in the neural network. However, this leads to learn a latent dense representation of the original input which may not be robust enough under several types of noise environment [3]. For such problems, [PERSON] et al. [27] developed a sparse deep auto-encoder model which can attain better robust features under noise compared to its non-sparse variant. By introducing a sparsity constraint to the reconstruction loss function, a better de-noising performance has been achieved. In fact, sparse representations have drawn much attention in different fields and they have a number of advantages in both theoretical research and practical applications [28; 29; 30]. Researches on sparse coding have demonstrated that the sparseness plays a key role in learning useful features [31; 32]. In sparse methods, the code is forced to have only a few non-zero units while most code units are zero most of the time. It has been found out in [15] that sparse representations learned in the context of auto-encoder variants are very useful in training deep architectures, especially for unsupervised pre-training of neural networks [33]. In particular, they have good robustness to noise and provide a good tiling of the joint space of location and frequency. Recently, a sparse auto-encoder based framework (SAF) [34] is proposed for SHM and shows a better performance in structural detection/identification in comparison with AutoDNet [3]. In their work, a Rectified Linear Unit (ReLU) activation function is used instead of 'tanh' which introduces sparsity effect in the neural networks.