# A Comparison of Segmentation Methods for Semantic OctoMap Generation
## 4 Results  
### Segmentation Mask Evaluation  
The test set results, summarized in Table 4, provide a comparison of four encoder-decoder-based architectures (U-Net [10], DeepLabv3+ [9], FPN [11], and Segformer [8]), each paired with different encoder backbones--MobileNetV4, EfficientNet-B2, and ResNet-34. Results are grouped into Loss and Accuracy Metrics. The objective of the comparison is to assess each model and encoder combination on its semantic segmentation quality. Across all evaluated architectures, EfficientNet-B2 consistently outperformed the other encoders, achieving higher overlap-based metrics such as Dice and IoU with low overall loss. For example, U-Net with EfficientNet-B2 attained a Dice score of 0.8392, an IoU of 0.6555, a test loss of 0.3608, a focal loss of 0.0240, and an F1-score of 0.7050. Similar improvements were observed for DeepLabv3+ and FPN when using the same encoder, confirming its reliability across different model architectures. The SegFormer architecture, when combined with EfficientNet-B2, delivered accuracy comparable to traditional CNN-based models such as FPN and DeepLabv3+, achieving a Dice score of 0.8376 and an IoU of \begin{table}
\begin{tabular}{l l c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Encoder**} & \multicolumn{3}{c}{**Loss Metrics**} & \multicolumn{3}{c}{**Accuracy Metrics**} \\  & & **CE** & **Focal** & **Total** & **Dice** & **F1-Score** & **IoU** \\ \hline \multirow{4}{*}{U-Net [10]} & MobileNetV4 & 0.296 & 0.037 & 0.541 & 0.792 & 0.660 & 0.601 \\  & EfficientNet-B2 & 0.176 & 0.024 & 0.361 & 0.839 & 0.705 & **0.656** \\  & ResNet-34 & 0.175 & 0.025 & 0.355 & **0.845** & **0.707** & 0.653 \\ \multirow{4}{*}{DeepLabv3+ [9]} & MobileNetV4 & 0.234 & 0.035 & 0.453 & 0.815 & 0.677 & 0.622 \\  & EfficientNet-B2 & 0.144 & 0.024 & 0.330 & 0.838 & 0.702 & 0.650 \\  & ResNet-34 & 0.155 & 0.023 & 0.342 & 0.836 & 0.697 & 0.646 \\ \multirow{4}{*}{FPN [11]} & MobileNetV4 & 0.237 & 0.033 & 0.457 & 0.814 & 0.690 & 0.635 \\  & EfficientNet-B2 & 0.145 & 0.024 & 0.330 & 0.838 & 0.697 & 0.646 \\ \multirow{4}{*}{SegFormer [8]} & ResNet-34 & 0.155 & 0.025 & 0.338 & 0.842 & 0.704 & 0.651 \\  & MobileNetV4 & 0.190 & 0.027 & 0.408 & 0.810 & 0.694 & 0.640 \\ \multirow{4}{*}{SegFormer [8]} & EfficientNet-B2 & **0.140** & 0.023 & **0.325** & 0.838 & 0.699 & 0.646 \\ \cline{1-1}  & ResNet-34 & 0.145 & **0.022** & 0.331 & 0.836 & 0.699 & 0.644 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Quantitative evaluation of semantic segmentation models on test set images. Metrics are grouped into loss functions (Cross Entropy Loss (CE), Focal Loss, and Combined Loss) and segmentation quality indicators (Dice, F1-score, and Intersection over Union (IoU)). \begin{table}
\begin{tabular}{l l l} \hline \hline
**Metric** & **Formula** & **Implementation** \\ \hline Intersection-over-Union (IoU) & \(IoU=\frac{\text{area}(P\cap G)}{\text{area}(P\cup G)}\) & MulticlassJaccardIndex() \\ F1-Score & \(F1=\frac{2\times\text{Precision-Recall}}{\text{Precision-Recall}}\) & MulticlassFIScore() \\ Dice Coefficient & \(Dice=\frac{2\sum_{k=1}^{N}|\left|\left|\left|P_{k}\cap G_{k}\right|\right.}{\sum_{k =1}^{N}(\left|\left|P_{k}\right|+\left|G_{k}\right|)}\right.\) & 1.0 - DiceLoss() \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Evaluation metrics. 0.6459. In contrast, all models using MobileNetV4 consistently underperformed in accuracy metrics. For instance, U-Net with MobileNetV4 reached only 0.7919 Dice, 0.6014 IoU, and had the highest test loss. When evaluated on downsampled images of 480 \(\times\) 304 pixels (Table 5), the U-Net with a ResNet-34 encoder achieves the highest overlap metrics Dice score of 0.7766 and IoU of 0.6193, outperforming DeepLabv3+ with EfficientNet-B2 Dice score of 0.7554 and IoU of 0.6012. Both U-Net (ResNet-34) and DeepLabv3+ (EfficientNet-B2) trained on SAM2-generated masks underperform compared to using SegFormer-B5-derived labels. For instance, U-Net's Dice (0.6913) and IoU (0.5543) are roughly 15 points lower than with SegFormer-B5 masks.