# Deep Learning-Based Applications for Safety Management in the AEC Industry: A Review
## 4 Challenges and Recommendations for Future Work  
### Limitations of the Data Set and Weak Generalization of DL Models  
To achieve a high detection or classification accuracy, normally a massive labeled image is needed to train a deep neural network well. However, there are few civil engineering-related data sets available to the public, and it is not easy to obtain such data. To this end, it is common to pre-train a CNN model with an extensive common object dataset (e.g., ImageNet, MNIST and CIFAR-10). Normally, most of the popular pre-trained CNN models (e.g., VGG-16, ResNet50, Inceptionv3, and YOLOv3) are available online. Such models with pre-trained weight have already learned to extract the basic features of the images. Researchers can download these models as the backbone of their DL structures for conducting specific tasks. This process is called transfer learning [20; 117], and many DL-based applications have employed it in practice. However, a pre-trained model is still required to learn the nuanced features of the specific targets. Given the lack of available datasets for training, researchers in construction are normally required to create their own by labeling images manually. This process is time-consuming, tedious and expensive. Since the specific database has to be created, it tends to be limited and, thus, can hinder the generalization ability of the model. To this end, data augmentation methods can be employed to upsize the dataset synthetically. In general, those methods include spatial flipping, cropping, bending and other deformations of the images [118]. Typically, the semantic meaning of the labels will not be changed by these augmentation approaches. Therefore, the newly generated images are highly associated with the original ones. In this way, the generalization of the DL model may not be improved effectively.  
In order to tackle these issues, a generative adversarial network (GAN) may be a possible option. GANs were first introduced by [PERSON] et al. [119] and can be used as data generation models. A GAN consists of a generator and a discriminator. The principle of a GAN is to let the discriminator evaluate the new data produced by the generator. As the recurrent training progresses, the performance can be improved gradually during the process. This kind of unsupervised learning has been proven to be powerful in many applications. In recent years, scholars start using GANs to produce synthetic images. Compared to the data augmentation methods, the images generated by GANs are more diverse and distinct [5]. In addition, researchers have used the advantages of GANs to generate face images. For example, [PERSON] et al. [120] applied a GAN to anime character creation. The GAN demonstrated stable performance in the innovative face creation of anime characters. [PERSON] et al. [5] designed a GAN which illustrated good outcomes in natural scene images. Besides those examples, [PERSON] and [PERSON] [121] leveraged a GAN in the health sector. They applied a deep GAN to produce synthetic prostate lesion images and utilized these images to upsize their training dataset. However, there are relatively few GAN-related studies conducted in the AEC safety industry. Implementing a GAN in practice might be a possible solution for tackling the construction-related data set challenge.  
Similarly, there are few public vibration data sets available for researchers to train their classifiers. Hence, future research should focus on finding an alternative method to train those classifiers when sensor data is insufficient. One recommended method for building up training data sets is combining real-life data from an undamaged structure with simulated data for the damage scenarios collected from an in-lab model. Besides that, as more and more emerging algorithms come out, unsupervised learning that can run on a small amount of labeled and unlabeled sensor data may become a hot topic in the future.