## 3 Results
### Kaarta Stencil
#### 3.1.2 Dataset Evaluation  
The values of standard deviation of the signed distances are respectively 7 cm for path 1 and 5 cm for path 2: these values, as well as the corresponding ones for the other two tested systems, summarize the discrepancies with respect to the reference dataset (see Table 2). This gives us a first overall indication of the accuracy obtainable by an IMMS surveying in similar conditions, although further statistical inference analysis should be carried out, for example based on robust non-parametric tests as suggested in ([PERSON] et al., 2016). Anyway, a relative comparison between the RMS values of the 5 different paths and/or IMSSs gives an idea of how each system works on average in the experimented conditions. Figure 7a and the corresponding figures for the other two systems display main discrepancies in three areas with trees, acquired in path 1: it is quite impossible to compute and evaluate the distances,and since the numerical values fluctuate between \(-1\) and \(1\) m they are meaningless for the test. Instead, among those surfaces with significant differences, the IMMS points with most errors are the farthest from the acquisition trajectory, such as points on the eaves, which are approximately \(15\) m high. Figure 6: Kaarta Stencil datasets: (**a**) path 1; (**b**) path 2. Figure 7: Signed distances of the Kaarta Stencil dataset compared to the ground truth: (**a**) path 1; (**b**) path 2. The data acquired with the Stencil are the least complete in our test, in particular for the vaults in the indoor areas. This is due to the fact that the system was held almost vertically, thus collecting data with a horizontal field of view of only 30\({}^{\circ}\). Therefore, in small and indoor spaces the upper part of the surfaces was not scanned. The system experienced some troubles when walking through narrow passageways, as was evident after the sequence of two doors (see details in the red circle in Figure 7a and recall the trajectory direction of Figure 2). A threshold distance was set to avoid scanning the operator walking with the system: probably the value of 2 m during our test was too high and it prevented data on closer surfaces from being recorded, which might have been useful. Figure 8 shows details of the horizontal section in those rooms and it highlights the increased misalignment after these passageways: up to 14.6 cm in the room with a large (170 cm \(\times\) 110 cm) brick pillar in the centre. The blue points (highlighted in the circle) show the relevant data in the elevation of the floor. When looking at the vertical section crossing this room (Figure 9), the problem is even more evident: the rotation of the shape of the floor is wrong, with a level arm effect affecting the resulting elevation of the floors, and error values of as much as 79 cm downwards and 68 cm upwards on either side of the pillar. More in detail, incorrect negative elevations arose in the left part, where the operator walked with the IMMS, while they are positive in the right part, which was surveyed with rays not hidden by the pillar. Our interpretation of these gross errors is that the instantaneous rotation values of the scanning device \(R_{tls}^{Scs}\) are not well corrected by the \"matching clouds\" algorithm when the points are extremely close together, as occurred when entering this room by means of a small gate of only 135 cm \(\times\) 140 cm in a 125-cm-wide wall, like a sort of \"small tunnel\". This rotation error affected the successive acquisition of the left part of the room, causing a large counter-clockwise roll error, while this was wrongly overcompensated by the LOAM algorithm for the right part of the room. Better agreement with the reference data (with discrepancies of less than 10 cm), is instead visible in Figures 7-9 in the \"open\" areas, for example, the octagonal room, whose \"entry tunnel\" gate measures 340 cm \(\times\) 330 cm \(\times\) 350 cm high.