# Implementation of Pavement Defect Detection System on Edge Computing Platform
###### Abstract  
Road surfaces in Taiwan, as well as other developed countries, often experience structural failures, such as patches, bumps, longitudinal and lateral cracking, and potholes, which cause discomfort and pose direct safety risks to motorists. To minimize damage to vehicles from pavement defects or provide the corresponding comfortable ride promotion strategy later, in this study, we developed a pavement defect detection system using a deep learning perception scheme for implementation on Xilinx Edge AI platforms. To increase the detection distance and accuracy of pavement defects, two cameras with different fields of view, at 70\({}^{\circ}\) and 30\({}^{\circ}\), respectively, were used to capture the front views of a car, and then the YOLOv3 (you only look once, version 3) model was employed to recognize the pavement defects, such as potholes, cracks, manhole covers, patches, and bumps. In addition, to promote continuous pavement defect recognition rate, a tracking-via-detection strategy was employed, which first detects pavement defects in each frame and then associates them to different frames using the Kalman filter method. Thus, the average detection accuracy of the pothole category could reach 71%, and the miss rate was about 29%. To confirm the effectiveness of the proposed detection strategy, experiments were conducted on an established Taiwan pavement defect image dataset (TPDID), which is the first dataset for Taiwan pavement defects. Moreover, different AI methods were used to detect the pavement defects for quantitative comparative analysis. Finally, a field-programmable gate-array-based edge computing platform was used as an embedded system to implement the proposed YOLOv3-based pavement defect detection system; the execution speed reached 27.8 FPS while maintaining the accuracy of the original system model. pavement defect detection; YOLOv3; tracking-via-detection; Taiwan pavement defect image dataset (TPDID); edge computing; embedded system +
Footnote †: journal: Applied  
out of control or even drift lanes due to a flat tire, causing a serious car accident. Moreover, the production boom of automated vehicles (AVs) is sweeping across the world. As of July 2019, there were 56 kinds of AVs in over 128 cities globally. Approximately 50% of these are shuttle buses for public transport. Besides safety, a crucial issue for the future of AVs is comfort, which is perhaps more important. Therefore, potholes on road pavements affect not only the structural safety of the vehicle but also the comfort of passengers in AVs. In severe cases, it can even cause vehicles to lose control and contribute to injuries and casualties. Potholes, manhole covers, patches, and cracks are among the pavement defects identified in this study using an imaging lens and a deep learning (DL) YOLOv3 (you only look once, version 3) model [3]. In addition, we employed the Kalman tracking algorithm to improve the reliability and detection rate of the system, which not only significantly reduces the miss rate and increases the average precision (AP) but also stabilizes the DL prediction frame for surface defects. Moreover, to identify potholes at a longer distance, we used two lenses with different fields of view (FoVs): 30\({}^{\circ}\) and 70\({}^{\circ}\) lenses. The 30\({}^{\circ}\) FoV lens is mainly used for identifying potholes at a distance. However, when a pothole is close to the vehicle, a pothole object may be too large to be fully represented in the image due to a small viewing angle, resulting in misjudgment or lack of identification. Therefore, we also employed the 70\({}^{\circ}\) FoV lens, which can be used to compensate for pothole detection at near and medium distances. Using these two lenses simultaneously can reinforce the reliability of the recognition results.