# Mixer U-Net: An Improved Automatic Road Extraction from UAV Imagery
## 4 Experiments, Results and Discussion
### Results and Analysis  
Figure 7 highlights the learning curves of all models during the training and validation process. In particular, Figure 7a,b elucidate the loss value, and Figure 7c,d portray mean IoU scores during training and validation, respectively. It can be noted from Figure 7 that U-Net and ResU-Net experience a slower convergence compared to other models while the proposed models yield the fastest convergence exceeding the 75% mean IoU benchmark easily within the first 10 epochs in the validation. However, in order to unlock the full potential of all models, we keep training them until performance is saturated. As a result, by the end of training, the proposed models remained dominant while the performance gap among models shrunk remarkably, scoring negligibly lower than 80% mIoU both on the training and validation sets. Table 1 summarizes the quantitative results of our experiments. Evidently, all models attained competitive results across all metrics in the inference time. Despite the fact that the proposed Mixer U-Net and Mixer U-Net with PN contain roughly 10-times fewer parameters than the baselines except ResU-Net, the models achieved the highest mIoU, recall, and F1 score, losing slightly to U-Net and ResU-Net in terms of precision. Among all models, FCN and DeepLabV3 yield inferior results, most probably, due to the reason that these models' backbone was pretrained on ImageNet-1000 dataset, the distribution of which notably differs from the target dataset. Although the ResU-Net model experiences relatively slower convergence at the beginning of the training, it achieves the highest results among the baseline models, surpassing the proposed models as well in terms of precision. Figure 7: Learning curves for the four baselines and the proposed models during training and validation. In all Figures, \(x\)-axis highlights the number of training epochs. In Figures (**a,b**), \(y\)-axis represents loss value, and in Figures (**c,d**), it describes the mean intersection over union (mean IoU) in percentage (%). It can be concluded that training models until full convergence might result in superior performance despite their initially slower convergence. Generally, all models yield good results with small distortion or misdetection. Figure 8 illustrates segmentation maps extracted by the models. Five images (row (a)) with varying difficulty were chosen randomly from the test set. The red dotted rectangles show areas where the models struggled the most, whereas green ones represent areas with relatively improved accuracy. On the first, third, and fifth images, all models struggle on the highlighted areas. However, on the second and fourth images, the proposed models extract nearly identical segmentation maps as the ground truth. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Models** & **mIoU** & **Precision** & **Recall** & **F1 Score** & **\# Params** \\ \hline FCN & 79.11 & 87.73 & 87.09 & 87.27 & 32.95 M \\ U-Net & 80.19 & 89.71 & 86.39 & 87.83 & 31.04 M \\ ResU-net & 80.64 & **89.76** & 86.7 & 88.01 & 8.22 M \\ DeepLabV3 & 79.28 & 88.36 & 86.48 & 87.27 & 39.63 M \\
**Mixer U-Net** & 80.39 & 88.73 & **87.31** & 87.87 & **3.19 M** \\
**Mixer U-Net with PN** & **80.75** & 89.42 & 87.02 & **88.04** & **3.67 M** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Quantitative results. Figure 8: Visual comparison of road extraction results by the baselines and the proposed models on randomly chosen images from the test dataset. The first two rows-(**a**,**b**) represent the original image and the ground truth, respectively. The order of models is as follows: (**c**) FCN, (**d**) U-Net, (**e**) ResU-Net, (**f**) DeepLabV3, (**g**) Mixer U-Net (ours), and (**h**) Mixer U-Net with PN (ours). Dotted red rectangles represent the areas of the image with poor performance, while dotted green rectangles highlight areas with better performance. In order to further explore the areas on images where the models struggle the most and the reasons for that, we conducted an error analysis (see Figure 9).