# Pavement Defect Segmentation in Orthoframes with a Pipeline of Three Convolutional Neural Networks
## 10 Experiments and Results
### DSN Architecture, Data Sampling and Patch Size Experiments  
With the flexibility of the U-Net approach, most ConvNet encoders can be used as the backbone for our DSN model. Many different pretrained ConvNets have been made publicly available, most notably models pretrained on the ImageNet competition [28]. While models that perform well on ImageNet generally do well on other datasets [29], this has not yet been verified for pavement distress related tasks. Thus, in an effort to search for the best performing encoder for our task, we evaluated a variety of pretrained encoders that have previously shown good results on the ImageNet dataset.  
The experiments were conducted on the 200 orthoframe dataset over 40 epochs, with 2-fold cross-validation. The encoders evaluated, provided by Reference [45], include ResNet34 [26], ResNet50 [26], SE-ResNet50 [46], Inception-v4 [47] and DenseNet-121 [48]. The training protocol for each model followed the one described in Section 10.2.3.  
From the results displayed in Figure 15, it can be seen that the Inception-v4 model outperforms other models both in terms of validation and training IoU. Therefore, we will proceed with this architecture in the future.  
To evaluate data sampling and patch size options that should be used with the full dataset, experiments presented in Table 2 were conducted on the 200 orthoframe datasets by training the DDN and DSN models with ResNet34 backbones for both. The DDN was trained on the full sampling strategy for each experiment that used a DDN. For result evaluation in Table 2, ground truths were used for road area detection.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline Dataset for DSN & Model(s) & Patch Size & Test Set IoU \\ \hline (1’) & ResNet34 DSN & 336 \(\times\) 336 cropped to 224 \(\times\) 224 & 0.275 \\ (1’) & ResNet34 DDN + ResNet 34 DSN & 336 \(\times\) 336 cropped to 224 \(\times\) 224 & 0.349 \\ (2’) & ResNet34 DDN + ResNet 34 DSN & 336 \(\times\) 336 cropped to 224 \(\times\) 224 & 0.349 \\ (3’) & ResNet34 DDN + ResNet 34 DSN & 336 \(\times\) 336 cropped to 224 \(\times\) 224 & 0.351 \\ (3’) & ResNet34 DDN + ResNet 34 DSN & 448 \(\times\) 448 cropped to 224 \(\times\) 224 & 0.344 \\ (3’) & ResNet34 DDN + ResNet 34 DSN & 224 \(\times\) 224 & 0.329 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Test set IoUs for different patch sizes and data sampling strategies on the reduced training set.  
Figure 15: Defect Segmentation Network (DSN) results for different encoder architectures over 40 epochs trained on the 200 orthoframe dataset. Performed with 2-fold cross-validation.  
Evidently, the DDN adds a lot of performance to the system, as the segmentation network alone trained with the full sampling strategy performs worse than systems using the DDN pipeline. The advantages of the \"classification before segmentation\" approach has also been noticed by other researchers [49].  
The difference between sampling strategies (2\") and (3\") seems negligible. While sampling strategy 1 achieved similar results when combined with a DDN, it was trained on nearly 3 times the amount of samples and the same amount of epochs. Thus, (1\") will be considered as an inefficient sampling strategy for the DSN and further experiments will proceed with sampling strategy (3\"). With regards to the patch size, performance gains can be noticed by adding more context to the patch and cropping the central part. However, it seems that adding context past the size of 336 \(\times\) 336 is unnecessary. Different levels of context for a patch are visualized in Figure 16.