## 2 Methodology  
### Analysis and Preprocessing of Source Data  
Closer observation of orthoframes that were collected with Ladybug 5+ in April 2019 revealed the following characteristics:  
\begin{table}
\begin{tabular}{c c  
1. Inconsistent sharpness across the image. This stemmed from the horizontal placement of some of the Ladybug cameras. Due to the simple laws of optics, road surface gradually loses detail as the distance from the original camera shooting location increases. See Figure 2.
2. Inconsistent brightness from image to image. This was related to the availability of light at the moment when an image was taken. Note that the situation has improved considerably not only because of the CMOS sensors of Ladybug 5+, but also because Reach-U Ltd. has instructed the MMSdriver to adjust the shutter speed manually during the data collection if the lighting conditions change on the road to avoid under- or over-exposure.
3. Comparatively high number of shadows cast by various objects on the road, near the road, by the camera rig, or by the car itself doing the mapping. The intensity of shadows is directly correlated with the availability of light, and their extent is (among other things) dependent on the angle of Sun rays, which is illustrated in Figure 3.  
In addition, there were various artifacts found in some images (vehicle fragments, people, etc.). However, there were only a few of these, so in general, they can be treated as statistically irrelevant and ignored.  
Figure 3: Shadows in the orthoframe cast by roadside objects and the vehicle body.  
Figure 2: Gradual loss of detail when the distance from the camera increases. The area above the white circle is considerably sharper than the rest of the image.  
Inconsistent quality of the images may mislead the ConvNet training [71], and although by forsaking the orthophoto format, we were able to avoid the \"stitching seams\" among individual orthoframes, the gradual loss of detail as we moved away from the position of the camera still presented a problem. We therefore focused on the sharper part of the orthoframe. The original orthoframe mask was multiplied with a filled circle with a radius of 1500 pixels. The resulting mask (Figure 4) was then used to extract the more detailed part of the orthoframe. As the consecutive orthoframes were overlapping, there was no loss of ground. Note also that in the resulting image, the area that was not road surface became much smaller and was most often present on one side of the road only and could be thus typically separated with a single line (Figure 5). Therefore, the road extraction procedure that was automated in [2] was now delegated to the digitizers (a digitizer in the context of this work refers to a person engaged in digitization of defect information based on the provided orthoframes and initial defect data) who processed the orthoframes as part of the preprocessing step for obtaining more accurate training data for the automatic detector.  
Out of 33,288 orthoframes, 20,318 that contained defects in the area of interest were algorithmically picked out for further consideration. Out of this selection, orthoframes with poor lighting conditions, poorly distinguishable defects, or other problems were abandoned after visual inspection. For actual digitization, 1572 orthoframes were used.  
Figure 4: Obtaining the mask for the more detailed part of the orthoframe.  
Figure 5: Due to the decreased area of interest, non-road surface is present only at the right side of the orthoframe.