# A Comparison of Segmentation Methods for Semantic OctoMap Generation
## 3 Methods
### Dataset  
Prior to data capture, we calibrated both the camera's internal parameters and its rigid body transformation relative to the LiDAR. For camera intrinsics, we used a checkerboard procedure: images of a known planar pattern were captured from multiple viewpoints, and OpenCV's pinhole calibration routines estimated the focal lengths, principal point, and lens distortion coefficients.  
To estimate camera-LiDAR extrinsic transform, we placed a flat calibration board with reflective targets (visible to both sensors) in various positions and orientations. Simultaneously recording LiDAR scans and undistorted camera images of that board, we extracted 3D points from the LiDAR returns on those targets and back-projected the corresponding 2D corner detections in the images into 3D rays. A least-squares minimization solved for the rotation and translation that best aligned the LiDAR point clouds with the 3D camera rays. We validated the resulting extrinsic parameters by projecting LiDAR points into the rectified image plane reprojection errors that remained below one pixel for all calibration targets. Our setup does not consider temporal calibration between the LiDAR and the camera like in [35].  
With intrinsics and extrinsics finalized, we collected the dataset on the outdoor premises of Poznan University of Technology using a remotely controlled vehicle (see Figure 2). The platform was equipped with an Ouster OS1-128 LiDAR, a calibrated Lucid Triton TRI064S-CC RGB camera, an Xsens MTi-30 AHRS, and DGPS (u-Blox F9P in moving-base mode).  
The platform was driven in separate runs across various areas of the campus to ensure diversity in the captured multi-sensor data. The recordings were saved as Robot Operating System (ROS2) bag files, enabling further processing. Two out of the recorded runs were selected and designated as the validation set (hereafter referred to as Validation scene) and the test set (hereafter referred to as Test scene). The validation route takes approximately 5 min, while the test route takes around 2 min. Figure 3 shows the trajectories followed by the robot during the collection of validation and test data. Since the images were recorded at a frequency of 15 Hz, resulting in the similarity between consecutive images of the sequence, we decided to keep only every 10 th image to train the neural networks. In the final split, the training, validation, and test sets consisted of 2418, 577, and 199 images, respectively. The images were additionally processed through distortion correction and rectification.  
\begin{table}
\begin{tabular}{c l c} \hline \hline
**ID** & **Category** & **RGB** \\ \hline
0 & Other & **(0, 0, 0)** \\
1 & Sky & (70, 130, 180) \\
2 & Building & (70, 70, 70) \\
3 & Grass & (152, 251, 152) \\
4 & Road & (128, 64, 128) \\
5 & Tree & (107, 142, 35) \\
6 & Traffic objects & (220, 220, 0) \\
7 & Car & **(0, 0, 142)** \\
8 & Person & **(255, 0, 0)** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Unified nine-class color map used in this paper.  
Figure 3: Comparison of vehicle trajectories on a campus map used to build annotated Octonaps. The blue line represents the test trajectory through the busy parking area near building 3C. The purple line shows the validation trajectory, conducted in a more complex and dynamic pattern near the Mechatronics, Biomechanics, and Nanoengineering Center.  
Figure 2: A remotely controlled vehicle platform used for data collection, equipped with an Ouster OS1-128 LiDAR (mast-mounted), a Lucid Triton TRI064S-CC RGB camera (front-facing), and DGPS antennas.  
In Figures 4 and 5, we show the example RGB and segmentation masks from the training and test sets. Despite being collected in a similar area, they contain significantly different objects, and the problem is challenging taking into account the generalization capabilities of the neural network and the complete mapping system.