# A Road Defect Detection System Using Smartphones
## 3 A Proposed Road Defect Detection System
### A Road Defect Detection with 1D-Cnn
To develop the YOLOv5m model used for data labeling, we utilized an open data source [45] and directly collected images, including 3000 for speed bumps, 3500 for manholes, and 4000 for potholes. The size of the YOLOv5m model is 882 MB. Through the data preprocessing step, we achieved 576 speed bumps, 290 manholes, and 271 pothole datasets with 100% accurate labels. However, 13 speed bumps, 68 manholes, and 69 pothole datasets were discarded as they could not pass the label quality verifier. The RDD-CNN model was constructed using TensorFlow 2.9.1. For model training, a total of 696 datasets were utilized, while 300 datasets were employed for testing purposes. In the input layer of the model, data consisting of 300 rows and a single column representing the RMS value of the accelerometer sensor data, forming a (300,1) dimension, was processed through a 1D convolution layer. In the output layer, the model was trained using the SoftMax activation function, the categorical cross-entropy loss function, and the Adam optimizer. The training process spanned over 30 epochs, with a batch size set to 15. Finally, the TFLite Converter from the Android TensorFlow Lite library was employed to generate a lightweight version of the RDD-CNN model. The optimized model is designed for execution on smartphones, including the Samsung Galaxy Note8, Xiaomi Redmi Note 10 Pro, and LG Q7, ensuring broad accessibility and practical deployment in mobile applications. ### Evaluation Results of the Automatic Data Collection System

The raw data undergo a process of threshold-based data classification and data slicing to isolate data segments containing road defects. In the threshold-based data classification, the acceleration threshold was set to 11 m/s\({}^{2}\) through iterative experiments to maximize the detection of road defects. Table 2 shows the discarded data ratio by the threshold-based data classification. Despite the presence of actual road defects, the results of undetected road defects by the threshold-based data classification are as follows: out of 604 speed bumps, 15 were not detected; out of 376 manholes, 18 were not detected; and out of 361 potholes, 21 were not detected. The 15 undetected speed bumps were all cases where the bump was simply painted on the road without any physical undulation. The undetected 18 manholes and 21 potholes were cases where the defects were not severe enough to cause significant vehicle vibrations. This highlights the limitation of threshold-based data classification, which relies on accelerometers and cannot classify defects with relatively mild undulations that do not cause noticeable changes in road surface level. Figure 4: Dataset collection location: (**a**) training dataset collection location; (**b**,**c**) test dataset collection location. Data exceeding the threshold value are subject to a data-slicing phase. Each sliced data segment comprises 300 acceleration data and 30 video images. To automatically generate labels for each sliced data segment, the 30 video images are analyzed using the YOLOv5m model. In an initial evaluation, the YOLOv5m model was applied to a comprehensive dataset consisting of 522 images, each collected by the front and rear dashcams during the day, and 356 images collected by the front dashcam during the night, for a total of 1400 images. The experiment was conducted during the day when there was no snow or rain, the sun was shining, or there were some clouds. The reason is that it is dangerous to collect data when it is snowing or raining, so it was conducted during the day when the weather was good. Table 3 shows significant performance gaps between day and night conditions, as well as between front and rear dashcams. The rear dashcam demonstrated significantly low performance in detecting road defects at night, warranting its exclusion from the table, while the front dashcam at night showed considerably low quality, as seen in Figure 5. The blue square represents the speed bump, the gray square represents the manhole, and the red square represents the pothole.