# Pavement Defect Segmentation in Orthoframes with a Pipeline of Three Convolutional Neural Networks
## 10 Experiments and Results
### Performance and Evaluation
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Iteration** & **Orthoframe \#** & **Model** & **MCC** & **Precision** & **Recall** \\ \hline Iteration 1 & 2206 & ResNet50 DDN & 0.687 & 0.856 & 0.621 \\ Iteration 2 & 3282 & ResNet34 DDN & 0.706 & 0.803 & 0.699 \\ Iteration 3 & 3583 & ResNet34 DDN & 0.713 & 0.753 & 0.761 \\ Iteration 4 & 5270 & ResNet34 DDN & 0.723 & 0.771 & 0.761 \\ Iteration 5 & 5842 & ResNet34 DDN & 0.716 & 0.730 & 0.793 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Defect detection metrics for the test set over iterations. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Iteration** & **Orthoframe \#** & **IoU** & **Precision** & **Recall** \\ \hline Iteration 1 & 2206 & 0.928 & 0.966 & 0.959 \\ Iteration 2 & 3282 & 0.949 & 0.973 & 0.970 \\ Iteration 3 & 3583 & 0.938 & 0.961 & 0.975 \\ Iteration 4 & 5270 & 0.955 & 0.976 & 0.978 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Road area segmentation metrics for the test set over iterations. as the seemingly ideal result only produces a pixel-wise IoU of 0.79 but an eloU of above 0.99. Figure 17c shows a number of false positives across the orthoframe, including one road marking in the middle of the road, a patch that is mistaken for weathering and various formations at the right side of the road. Additionally, the road area segmentation task is made harder by the crossroad. In Figure 17d, we see sharp shadows across the orthoframe, which do not generate false positives on their own. However, despite the well segmented alligator cracking, the right road edge suffers from false negatives and false positives, which might be partly due to the image being blurrier in that part of the image. Figure 17: Orthoframes from the test set processed by the PDDS. RSN outputs multiplied by the region of interest mask are highlighted. Patches classified by the DDN as non-defective have white borders and patches classified as defective have red borders. For the masks produced by DSN, the green, red and blue contours represent true positive, false positive and false negative pixels respectively. (**a**) IoU = 0.846, eloU = 0.930, iloU = 0.457, cPr = 0.600, cRC = 0.750; (**b**) IoU = 0.790, eloU = 0.991, iloU = 0.582, cPr = 1, cRC = 1; (**c**) IoU = 0.104, eloU = 0.110, iloU = 0.306, cPr = 0.143, cRC = 1; (**d**) IoU = 0.515, eloU = 0.575, iloU = 0.306, cPr = 0.750, cRC = 0.500. Lastly, as the system is comprised of three models of around 100 million parameters in total and there are thousands of kilometers of roads to cover, we must consider the inference speed as an important performance metric. In Table 7, inference speed averages over the test set are provided. These experiments were conducted on a Windows 7 PC with an Intel i7-7700 CPU, 16 GB of RAM and a GeForce GTX 1080 GPU. Note, however, that multiprocessing was not used in these experiments, so further optimizations are possible. For the test set, there are on average about 63 patches per orthoframe and on average 11 of them get detected as defective. Thus, the DDN takes 33 ms per patch and the ResNet34 and Inception-v4 networks take 59 ms and 79 ms per patch, respectively. As the average orthoframe covers around 3 m of road in length, the processing speed is about 3 m/4.85 s \(\approx\) 0.62 m/s.