# A Comparison of Segmentation Methods for Semantic OctoMap Generation
## 4 Results
### Semantic OctoMaps Validation  
To compare the quality of segmentation in the 3D space, we processed recorded sequences from rosbag files through the ROS2 node while employing the complete set of previously trained models. We set the OctoMap resolution to 25 cm, enabling the creation of a large yet relatively precise representation of the environment. For analysis, we obtained a point cloud formed by points located at the centers of occupied voxels in the map. Approximately 16% of points were removed in both scenarios as they were not assigned a color during the mapping process (these points maintained the default pure white color because the voxel does not have the required number of measurements--in our implementation, a point must be located inside the voxel in two separate scans). To assign a point to a particular class, its color values must not differ by more than 5 units in any channel from the reference color values listed in Table 1. We define non-classified points (NC) as those which, despite having assigned color values, could not be assigned to any semantic class by not meeting the color difference criterion. This stems from color blending in the OctoMap when a particular spatial region was repeatedly annotated with different labels. In such a case, there is insufficient confidence for the point to be assigned to any semantic class. To establish ground truth data, scene elements were manually annotated with cuboids. We split the experiment into several phases. First, we evaluated segmentation performance using all model-encoder combinations using the default image resolution. Subsequently, we extended the analysis with a comparison using downscaled images and maps created by models trained on masks produced by Grounding DINO and SAM2. The results are presented in Tables 9, 10, and 11, respectively. These tables store the following metrics: accuracy (ACC), weighted F1-score (wF1), where the weights are directly proportional to the fraction of points of each class in the complete map, and the percentage of unclassified voxels (NC). The index \({}_{\text{f}}\) appended to the metric name indicates that the value was computed for the subset of points after filtering out those not assigned to any class. Finally, we compared F1-scores achieved for individual classes by top-performing models for each encoder type across all three experimental setups. The metrics are visualized in the graphs shown in Figures 6-8. In Figures 9 and 10, we present manual ground truth labels alongside OctoMaps generated using selected segmentation approaches. As shown in Table 9, FPN coupled with the lightweight MobileNetV4 backbone achieves the highest accuracy on validation scene \(\text{ACC}\approx 0.7988\), filtered accuracy \(\text{ACC}_{\text{f}}\approx 0.9365\), filtered weighted \(F_{1}\)-score \(F1_{\text{f}}\approx 0.8627\), and low unclassified rate \(\text{NC}\approx 0.1471\) on the validation scene. It also generalizes well to the test scene, with \(F_{1,f}\approx 0.7934\) and \(\text{ACC}\approx 0.7322\), indicating that hierarchical feature integration with a lightweight encoder can outperform other configurations in both accuracy and efficiency. On the other hand, DeepLabv3+ paired with EfficientNet-B2 not only achieves the lowest unclassified-voxel rate on the test split (\(\text{NC}\approx 0.1148\)) but also maintains competitive overall performance (\(\text{ACC}\approx 0.7369\), wF1 \(\approx 0.7976\)). This configuration demonstrates also an optimal balance between reducing unclassified areas and maintaining high segmentation accuracy. Furthermore, U-Net with a MobileNetV4 backbone delivers strong performance,particularly on the test scene, where it achieves the highest accuracy among all combinations (ACC \(\approx\) 0.7383) and the highest filtered weighted \(F_{1}\)-score (wF1\({}_{f}\)\(\approx\) 0.8599). On the validation scene, it records ACC \(\approx\) 0.7927, ACC\({}_{f}\)\(\approx\) 0.9336, wF1\({}_{f}\)\(\approx\) 0.9369, and an unclassified-voxel rate of NC \(\approx\) 0.1509.