# Towards Low-Cost Pavement Condition Health Monitoring and Analysis Using Deep Learning
## 2 Materials and Methods
### The Use of Deep Learning in Pavement Engineering  
Given the advances of deep learning, there has been significant research using these techniques for Pavement Engineering applications [21; 22; 23; 24]. These applications can be assigned to the following areas: Pavement condition and performance predictions [25; 26; 27; 28], Pavement management systems [29; 30; 31], pavement performance forecasting [32; 33; 34], structural evaluations [35; 36; 37], modelling pavement materials [38; 39; 40] and pavement image analysis and classification [22; 41; 42; 43]. Pavement Image analysis and classification is the most researched area, where the focus has been split between image classifications, where images are classified based on the distress occurring in the image; and object detection, where distresses are located within bounding boxes or masks within the image. There are however issues with image classification as distresses regularly occur in a grouped manner thereby making it difficult to label a particular image with just one distress type label. This is important for a road engineer as information on connected distresses and the locations of areas where there are multiple distresses is vital for the asset management system to accurately monitor the road conditions. With object detection, it is possible to have multiple overlapping objects within an image and therefore it possible to detect multiple overlapping pavement distresses. A significant proportion of previous studies has been focused on developing models and networks to determine whether there is a presence of a distress or not and also the general detection of pavement distresses [41]. Of the distress types, the main focus has been crack detection and analysis. This is due to the fact that pavement cracks are seen as the most predominant distress type [45] and they are also easier to measure, with the typical requirements being simply to measure the crack's width and length. There are a tremendous number of studies on developing specific neural networks for crack detection and analysis using both 2D and 3D imagery [41; 42; 46; 47; 48; 49; 50; 51; 52] and with comparisons made to results from image-based toolboxes for crack detection and analysis such as CrackIT [53]. While the detection and monitoring of cracks are important to road agencies, this represents only one main category of distress. There has not been a lot of focus on standardized distress categories as accredited by international manuals on the subject [39]. There have been a few that have tried to analyze multiple distress categories and generate datasets of multiple types. A research team in Germany developed a CNN for pavement distress application based on imagery obtained through surveys across the German road network using a mobile mapping system attached to a vehicle [54; 55]. Their team developed,  
Figure 2: Typical model development setup. the German Asphalt Pavement Distress (GAPS) dataset which was utilized to generate classifications based on six different distress categories based on the German road manuals with research ongoing utilizing their developed Neural network, ASVINOS. Studies have also been carried out in Italy in which fourteen different categories of distresses were analyzed with the application of semantic segmentation and object detection algorithms on a dataset within Naples, Italy [56]. This was done to formulate a decision support system based on the occurrence of the predicted distresses within the datasets which further highlights the importance of the detection of multiple distresses. There is also a large dataset of road surfaces which exists called the KITTI dataset [57] but this dataset was created primarily for the purpose of assisting with automated driving research. There has also been the development of a database of road distresses in Japan through a mobile application [58] in which eight distresses were annotated within the dataset.