# Towards Low-Cost Pavement Condition Health Monitoring and Analysis Using Deep Learning
## 3 Materials and Methods
### Data Annotation  
For this study, the open-source labelling software, LabelImg [63], was utilized to individually manually label images based on the type of distresses present. An example of this work is shown in Figure 5. This was done by trained civil engineers with experience in asphalt pavement engineering and conditional surveys, which are typically used to define and detect pavement distresses. Figure 4: Image depicting smartphone setup during surveys. Figure 5: Annotation of images using LabelImg. This software utilizes the PASCAL VOC format [64] for the labelling to create. xml files for each image. The critical and novel step in this process was the development of the distress categories for the model. Typically, pavement distresses can be broken down into four major categories namely: cracking, visco-plastic deformation, surface defects and other miscellaneous types [13]. The grouping is shown in Table 1.  
Of these categories, other studies have shown that there is a direct relationship between the impact each distress has upon safety and comfort per level of severity [13]. Based on this each distress has an impact based on the severity of the occurring distress. This is further depicted in Figures 6 and 7. From these figures, it can be seen that the most impactful severity level, as expected, was the High severity cases whilst the medium and low severities in most cases have fewer impacts upon safety and comfort. The severity level is usually then derived based on data from manual surveys and based on regulations implemented by pavement distress manuals such as [65; 66; 67]. Different ratings are then utilized to determine the overall condition of the roads based on the presence of the distresses and other factors such as roughness which are correlated in typical performance indices such as International Roughness Index (IRI) [68] and Pavement Condition Index (PCI) [65]. Based on this, the decision was chosen to have only two severity levels (level 1 and 2), wherein the first represents situations wherein remedial action is not a necessity and the second where it should be done. Using the four general groups as a base case, annotations were made for each with the exemption of Surface defects (Bleeding, Polished aggregate and Raveling) where it is difficult to accurately pinpoint the distress and its associated severity level from a 2D image. The cracking group was also split between general cracking (gc) and area cracking (ac) (as defined by cracking made over a section, such as alligator cracking, as opposed to instances of cracks that are formed at specific points on the road surface or along its surface). With these categories, the developed model should be able to predict different instances of cracking, instances of visco-plastic deformations (vp) and miscellaneous distresses (msc) such as manholes whilst also providing a quick analysis of their severities. There are limitations using this approach as no precise metric measurements would be done on the distresses and having only 2 severity levels does not provide information on cases where intervention action may be required in the near future as could be interrupted by a medium level severity assessment. However, gaining a general understanding of where these groups of distresses occur and their frequency can provide practitioners with valuable information and allow for an adequate resource for continuously monitoring the overall health of the road structure. Future work will focus on how to integrate other types of groupings as well as surface defects. From these groupings, annotations were manually made only on images where there were was a clear view of the distress and it could also be clearly marked. This resulted in a total of 4862 distress annotations to be used for the model as split as shown in Figure 8.  
Within this distribution, the most observed distresses are general cracking followed by viscoplastic deformations and then area cracking, with the least being the miscellaneous category.