# Automatic Detection of Ballast Unevenness Using Deep Neural Network
## 2 Algorithm for Automatic Detection of Unevenness in Ballast  
The poor condition of ballast has a significant impact on both traffic safety and travel comfort. The unevenness of the ballast surface is one measure of the condition of the ballast. The algorithm presented in this paper allows us to determine the unevenness of the ballast. According to Polish railway line standards, when ballast unevenness is within \(+/-\) threshold = 35 mm, it does not pose a threat to traffic safety. These levels are measured relative to the top surface of the sleepers. Figure 1 presents a cross-section of the track with marked thresholds.  
The state of the railway track is monitored using a track inspection vehicle managed by the Polish railway lines--Figure 2. All available data used in our research were collected using this vehicle. One of the several measurement systems installed on the board of the vehicle is a laser triangulation system. It comprises pulsed high-power laser line projectors and synchronized cameras. This system captures a high-resolution intensity image and 3D range profile of the railway track. Laser light is used to illuminate railway surfaces and high-speed cameras are used to capture images of the projected light including its intensity. Figure 3 shows a block diagram of the laser triangulation system. Such a system captures both Intensity and Range images of the railway track simultaneously. Intensity images are produced by mapping the intensity of the reflected laser light, and Range images (depth map), marked in this figure as Dr, are produced by mapping the elevation of each measurement point. The pixels of these two maps correspond one-to-one, that is, the corresponding image regions in the intensity image appear at the same position in the depth map. If a certain target is detected in the intensity map, the corresponding target can be found at the same position in the depth map (Range image).  
Figure 4 shows a block diagram of the ballast unevenness detection algorithm. It is composed of two components. The first component presented in this figure as the \"detection of sleepers\" box is responsible for the detection of sleepers in the track and is described in Section 2.1. It uses the intensity image coming from the laser triangulation system as an input. Deep neural networks for instance segmentation are used for sleeper detection (both wooden and concrete). The second component presented in this figure as the \"detection of ballast out of range \(+/-\) Threshold\" box specifies the areas of ballast for which its height is outside the defined range of the \(+/-\) threshold = 35 mm and is described in Section 2.2. It uses the range image (map) coming from the laser triangulation system as an input and sleeper masks generated by the first component. The width and the height of both the intensity and the range image are equal to 288 and 1024, respectively.  
Figure 1: Cross-section of the track with marked threshold.  
Figure 4: Block diagram for the proposed algorithm.  
Figure 3: Block diagram for a laser triangulation system.  
Figure 2: Track inspection vehicle managed by Polish railway lines.