# Fine-Grained Detection of Pavement Distress Based on Integrated Data Using Digital Twin
###### Abstract  
The automated detection of distress such as cracks or potholes is a key basis for assessing the condition of pavements and deciding on their maintenance. A fine-grained pavement distress-detection algorithm based on integrated data using a digital twin is proposed to solve the challenges of the insufficiency of high-quality negative samples in specific scenarios An asphalt pavement background model is created based on UAV-captured images, and a lightweight physical engine is used to randomly render 5 types of distress and 3 specific scenarios to the background model, generating a digital twin model that can provide virtual distress data. The virtual data are combined with real data in different virtual-to-real ratios (0:1 to 5:1) to form an integrated dataset and used to fully train deep object detection networks for fine-grained detection. The results show that the YOLOv5 network with the virtual-to-real ratio of 3:1 achieves the best average precision for 5 types of distress (asphalt pavement MAP: 75.40%), with a 2-fold and 1.5-fold improvement compared to models developed without virtual data and with traditional data augmentation, respectively, and achieves over 40% recall in shadow, occlusion and blur. The proposed approach could provide a more reliable and refined automated method for pavement analysis in complex scenarios. road engineering; pavement-distress detection; digital twin; integrated data; physical engine; deep-object detection network +
Footnote †: journal:  
to complex detection scenarios such as noise and uneven lighting [13]. Machine learning approaches are based on training classifiers such as support vector machines (SVM), naive Bayesian classifiers (NBC), and artificial neural networks (ANN) to identify different types of pavement distress by learning limited and perceptible features [14; 15; 16; 17]. [PERSON] et al. [18] used and revised the BP neural network for pavement surface crack image processing and utilized the self-studying feature of the network to identify cracks. [PERSON] et al. [19] proposed three SVM models for classifying five types of pavement cracks to intelligently identify cracking types in an automated manner. These methods convert image recognition to probabilistic assessment; However, the extraction of these features also relies on prior knowledge and engineering experience, which inevitably reflect the high specificity, low generality and complexities of image processing-based techniques [20; 21]. Furthermore, pavement distress images have the characteristics of small differences between the background and the target, and are always with noise, making feature recognition difficult. Also, it is difficult to guarantee the accuracy and consistency of detection results for pavement crack images with a complex topology, which require some targeted and refined algorithms to be designed for specific scenarios. The representatives of deep learning methods, convolutional neural networks (CNN), can automatically extract rich, highly abstracted features from the inspection data to achieve the accurate detection of pavement distress [22; 23; 24; 25; 26; 27; 28]. [PERSON] [29] proposed a CrackFormer framework for pavement-crack detection, which employs a transformer-based high-resolution network architecture to gain a multi-scale feature semantics of cracks, achieving an overall precision exceeding 90% as tested on seven crack datasets. [PERSON] et al. [8] transferred the parameters and fine-grained the last layers of the VGG-16 DCNN pre-trained on the ImageNet to solve the crack detection problem of an asphalt and concrete pavement, a process that achieved excellent detection results in a classification study. [PERSON] et al. [30] designed a CrackNet model to detect cracks in 3D asphalt pavements based on images captured by pavement digital inspection vehicles, resulting in an increase in accuracy of over 20% compared to conventional 3D shadow modelling and the HOG feature-based SVM method. [PERSON] et al.