# A Review of Deep Learning Applications for Railway Safety
## 4 Railway Infra Safety
### Rail Surface  
Scouring, breaking, and deficient fastening in bolts and sleepers are typical defects on the rail surface. Figure 3 presents several types of rail surface defects. [PERSON] et al. [45] proposed a machine learning model based on image features of defects extracted based on PCA, kernel principal component analysis (KPCA), singular value decomposition (SVD), and histogram match (HM). [PERSON] et al. [46] adopted deep convolution neural networks to determine defect types of surface images (normal, weld, L-squat, M-squat, S-squat, and joint). They designed and compared three CNN models (small, medium, and large), each with different structures (number of layers, number of filters, sizes of filters, activation functions). The large model outperforms small and medium models and shows about 93% accuracy in detecting surface defects. Many studies have been performed to develop object detection methods on rail surface images. For example, [PERSON] et al. [47] developed a fast and accurate defect detection model for rail surfaces using YOLO v3, which has the strength of accurately and quickly detecting small-sized targets. Figure 2: Catenary Inspection Using the Faster R-CNN network. Reproduced with permission from Elsevier [39]. The detection model receives 416 \(\times\) 416 images and divides them into boxes of various sizes, calculates normalized coordinate values of defects depending on the location of defects located inside the box, predicts defect inclusion scores for each box, and evaluates reliability. This method achieved high detection rates (97%) in 0.15 s. Similarly, [PERSON] et al. [48] developed a model that detects the location of defects from existing rail surface images. Their proposed model consisted of a MobileNetV2 for extracting image characteristics and a YOLOv3 module for defect localization. Their performance test results confirmed that the model increased the mean average precision (MAP) by more than 4%. [PERSON] et al. [49] presented a novel pipeline consisting of two stages. In the first stage, an input image is localized to extract rail areas. The second stage detects defect areas using a deep learning model, a fine-tuned Inception3. Some studies proposed deep learning methods to extract defects more detailedly using image segmentation. [PERSON] et al. [5] adopted image segmentation to distinguish specific areas of defects on rail surfaces. The defective part was labeled in units of image pixels to train the segmentation model. The proposed model was implemented based on the VGG-19 structure and showed IoU and F1 scores exceeding 90%. [PERSON] [50] proposed SegNet, a deep convolution neural network, to detect defects on rail surfaces. As shown in Figure 4, SegNet comprises feature extraction (FE) and feature construction (FC). This structure can learn rail surface types and their distributions from a given training dataset. Figure 3: Several types of rail surface defect images. Reproduced with permission from IEEE [47]. [PERSON] et al. [8] proposed a technique for detecting rolling-contact fatigue (RCF), which is a failure or material removal driven by crack propagation caused by a near-surface alternating stress field. Specifically, this study used laser ultrasound scanning data to detect RCFs. To extract features from ultrasonic signals, wavelet packet transform (WPT), which decomposes signals in different frequency bands, and KPCA, which reduces the correlation between all defective features, were used. A support vector machine (SVM) model performed the final detection based on the features. A squat is an RCF defect and often leads to rail breaks. [PERSON] et al. [51] proposed an algorithm to automatically detect the position of rail squats using vehicle axle box acceleration signals. The convolutional variable auto encoder (CVAE), an unsupervised manager, extracts critical features from signals, and the one-class SVM (OCSVM) detects rail squats in abnormal conditions. In their study, the proposed method was shown to be robust to signal noise and train speed variability.