# Pavement Defect Segmentation in Orthoframes with a Pipeline of Three Convolutional Neural Networks
## 1 Introduction  
Pavement distress detection is one of the most important topics in pavement management [1]. Since pavement tends to deteriorate with time under the influence of traffic and environmental variations, efficient and timely road inspection is one of the key elements of a successful pavement management system (PMS) and condition of roads must be monitored routinely. Pavement distress evaluation has significantly developed from manual visual surveys [2]. Currently, pavement images are often obtained automatically, using high-speed digital cameras attached to data collection vehicles, enabling them to move at highway speeds [3]. The obtained digital images of pavement surfaces are processed and stored, the crack features are extracted and recorded in the pavement distress database, which can be linked to a PMS. Other technologies, for example, LiDAR [4] and laser [5] based systems have also been considered but they have one important drawback--most such systems operate at relatively low speed, which increases time and cost of data acquisition and decreases road traffic safety [6]. Recent advances in artificial intelligence (AI), in the field of computer vision, in particular, have led to the development of automated image-based pavement distress detection methods [7, 8]. By now, the classical intensity-thresholding, edge detection and even the more advanced machine learning methods (such as Reference [9]) have largely lost the battle to Deep Learning (DL) Convolutional Neural Networks (CNNs) [10, 11, 12, 13, 14, 15, 16]. Depending on the chosen approach, the defect detection can be performed at different levels of precision. For example, Reference [17] localizes potholes on panoramic images by drawing a boundingbox around the potholes. In contrast, Reference [12] uses binary classification--defect or no defect--on the input without providing a location of the defect; an approximate location of the defects is obtained by dividing an orthoframe into smaller section on which the binary classifier is applied. In order to obtain pixel level accuracy, semantic segmentation is necessary to determine the exact shape of the defect. For example, Reference [18] uses a method similar to sliding window method for semantic segmentation while selecting images for annotation based on uncertainty estimation and Reference [19] uses a U-Net based ConvNet approach by incorporating a global context block to enhance performance. Despite all the development, pavement distress detection is yet a challenging problem and the abilities of existing methods are somewhat limited, largely due to the inherent difficulties associated with pavement images (inhomogeneity of defects, diversity of surface texture, presence of non-crack features such as joints, etc.) [20]. The pavement distress detection workflow therefore still contains a substantial amount of manual work in practice. The operator only takes advantage of the tools built into the defect detection software to better digitize the defects, conduct measurements and report results or--in the semi-automatic approach--manually adjusts incorrect results that are provided by the pavement distress detection algorithm. In Estonia where the distress on roads is accelerated by temperature shifts (the daily temperature can fluctuate around 0\({}^{\circ}\)C for more than five months a year), EyeVi Technologies Ltd--a spin-off company from Reach-U Ltd, specializing in on-demand mapping technology and AI feature extraction--has developed a fast-speed mobile mapping system (MMS) employing six high-resolution cameras for recording images of roads [21]. Unusually, none of these cameras is effectively looking directly downward at the road pavement--the orthophotos are derived from collected panoramic images in post-processing--and the system is not equipped with a high-performance lighting unit, making the photos susceptible to exterior lighting conditions and shadows cast by various roadside objects.