# Automatic Detection of Ballast Unevenness Using Deep Neural Network
###### Abstract  
The amount of freight transported by rail and the number of passengers are increasing year by year. Any disruption to the passenger or freight transport stream can generate both financial and human losses. Such a disruption can be caused by the rail infrastructure being in poor condition. For this reason, the state of the infrastructure should be monitored periodically. One of the important elements of railroad infrastructure is the ballast. Its condition has a significant impact on the safety of rail traffic. The unevenness of the ballast surface is one of the indicators of its condition. For this reason, a regulation was introduced by Polish railway lines specifying the maximum threshold of ballast unevenness. This article presents an algorithm that allows for the detection of irregularities in the ballast. These irregularities are determined relative to the surface of the sleepers. The images used by the algorithm were captured by a laser triangulation system placed on a rail inspection vehicle managed by the Polish railway lines. The proposed solution has the following elements of novelty: (a) it presents a simple criterion for evaluating the condition of the ballast based on the measurement of its unevenness in relation to the level of the sleeper; (b) it treats ballast irregularity detection as an instance segmentation process and it compares two segmentation algorithms, Mask R-CNN and YOLACT, in terms of their application to ballast irregularity detection; and (c) it uses segmentation-related metrics--mAP (Mean Average Precision), IoU (Intersection over Union) and Pixel Accuracy--to evaluate the quality of the detection of ballast irregularity. deep learning; object detection; image segmentation; railway track diagnostics +
Footnote †: journal: Medical  
the Dielectric Constant Method and 2 GHz GPR to evaluate ballast. In [9], the ballast state is assessed based on the GPR signal waveform and wavelet transform. Since the conversion of electromagnetic signal characteristics of GPR to ballast fouling is performed indirectly, the main drawback of methods based on GPR is the susceptibility to electromagnetic interference. It causes problems if the railway runs through areas that contain special structures such as turnouts, bridges with rail guards, etc., as the signal cannot be correctly detected because of iron interference. Other methods that were used in the assessment of ballast are based on vibration measurements. Paper [10] demonstrates the detection of the damage status of ballast under a sleeper through the monitoring of the vibrations of the corresponding sleeper. The vibration is generated through a simple impact hammer test. Another paper [11] explored the feasibility of using particle acceleration responses to diagnose mud-pumping ballast. It used innovative wireless sensors with 3D-printed shells resembling the real shape of ballast particles. These sensors are placed in the ballast and measure acceleration. According to [11], it seems promising to use particle-scale acceleration underneath tie plates as readily implementable indicators for smart in-service track health monitoring. The main drawback of these methods is that the measurement is made at fixed points in the track. Another paper [12] attempted to use infrared thermography to detect fouling ballast. It investigated the impact of the intensity of solar radiation and rainfall on the surface temperature of the ballast. The experimental results showed that noticeable temperature differences between clean and fouled ballast occur on sunny days and after rainfall. In contrast, the differences are imperceptible at nighttime. Another type of device that can be used in ballast diagnostics is LIDAR (Light Detection and Ranging). It consists of a laser illuminating the track and a camera recording the image of the reflection from the ballast (surface map) [13; 14]. Article [15] compared the performance of a low-cost solid-state LiDAR system with a traditional static LiDAR system.