## 2 Proposed Method
### Voxel-Based Scene Representation
Since the ground truth camera poses are included in the training dataset, the coordinates (x and y) of the inlier 3D points in the image can be computed using geometrical reprojection. By applying the convex hull operation to the 2D point set corresponding to the inlier 3D points, we can obtain an image region for each inlier voxel. As the convex hull of the inlier set is the \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**King’s College**} & \multicolumn{3}{c}{**Old Hospital**} & \multicolumn{3}{c}{**Shop Facade**} & \multicolumn{3}{c}{**St Mary’s**} \\  & & & & & & & & & & & **Church** \\ \hline Number of voxels & 50 & 100 & 120 & 15 & 20 & 30 & 20 & 30 & 40 & 50 & 100 \\ Avg. number of & & & & & & & & & & & \\ inlier 3D points & 426 & 323 & 271 & 295 & 195 & 139 & 299 & 197 & 139 & 995 & 403 \\ per voxel & & & & & & & & & & & \\ Avg. distance & & & & & & & & & & & \\ errors (cm) & 3.3 & 3.4 & 3.5 & 3.4 & 3.4 & 3.4 & 3.1 & 3.0 & 3.4 & 3.3 \\ Number of & 86 & 140 & 194 & 120 & 229 & 318 & 95 & 157 & 241 & 150 & 474 \\  & & & & & & & & & & & \\ \hline Total number & & 1563 & & & 1077 & & & 334 & & 2017 \\ of images & & & & & & & & & & & \\ Spatial Extent (m) & & \(140\times 40\) & & & \(50\times 40\) & & & \(35\times 25\) & & \(80\times 60\) \\ Number of & & & & & & & & & & & \\
3D Points & & & & & & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Summary of the key scene representation metrics for the scenes from the Cambridge Landmarks dataset [3], according to the number of voxels. smallest convex polygon that contains all the inlier points, the apparent contradiction of requiring multiple voxels in a small region can be addressed. In the training data structure, inlier image regions are labeled with the identification number of the voxel. Figure 2: **(a)** Voxel-based representation of the Shop Facade scene. **(b)** Inlier 3D point set for the Shop Facade scene after random sample consensus (RANSAC)-based plane fitting. **(c,d)** Top views of **(a,b)**, respectively. **(e)** Voxel-based representation of St Mary’s Church scene. **(f)** Inlier 3D point set for St. Mary’s Church scene. **(g,h)** Top views of **(e,f)**, respectively. **(i)** Voxel-based representation of the Heads scene. **(j)** Inlier 3D point set for the Heads scene. In this study, labeled regions in an input image were detected using AdapNet, originally proposed for semantic segmentation with complementary modalities and spectra, such as RGB and depth [26; 27]. Here, AdapNet was trained on the indexed images with the voxel's identification number. In other words, the residual learning model for semantic segmentation is trained to detect the image regions labeled with the voxel's identification number. As AdapNet is based on ResNet, it includes batch normalization and skip convolution layers. In our experiments, we changed the dimensions of the convolutional filter before the skip connection, modifying two feature maps of the size \(12\times 48\times 96\) to two feature maps of the size \(64\times 48\times 96\). As CNNs require large amounts of training data, the ResNet50 model was initially pretrained using the ImageNet [28]. Following this, AdapNet learns landmark features in each voxel region, their spatial distribution, and the alignment of voxel regions in our training dataset. To minimize false detection of the voxel regions, these are examined based on their spatial relation to ensure that the geometric consistency of the scene representation, such as left-right order, is satisfied. Here, we assume that no camera rolling (rotation in the z-axis direction) occurs. Voxels in the scene are assigned a unique identification number sequentially, according to their location in the initial scene representation setup. This enables us to reference the original scene when examining the geometric relationship between voxels, such that falsely detected regions can be deleted. Voxel regions with areas that are too small are also removed.