## 4 The M_HERACLES Toolbox
### Step 2 (1): Automatic Segmentation and Classification of Structural Supports
#### 4.2.2 Results and Analysis
The precision level is low for the Valentino, at 66.68%, which suggested overclassification. As has been mentioned before, this is mainly due to the ceilings of the dataset that represent arcs instead of planar surfaces as had been hard-coded in the algorithm. A further improvement of the algorithm may incorporate this possibility into account, as this type of ceiling can be found in many heritage datasets. The recall value is, however, quite high, with a median value of 88.16% and thus yielding an F1 score of 75.92%. This means that the algorithm does nevertheless give promising results. Indeed, in some applications where high precision is not necessarily required (e.g., training data generation for deep learning techniques), these results may be sufficient. As far as the classification goes, Figure 10 shows that the algorithm managed to correctly classify the pillars of the Kasepuhan dataset as non-columns. Indeed, under the definition of columns as set in this paper and contrary to classical columns, the eight structures in the Kasepuhan dataset cannot be classified as columns as they are in fact rectangular shaped posts. For the St-Pierre, the algorithm correctly determined that the eight detected structures are inside the \"column\" class. In the case of the Valentino, it also managed to correctly identify that the free-standing pillars are columns, while the rest of the detected structures were classified as non-columns. The processing time of the datasets showed that they may be, at least in part, linked to the number of points inside the input data. However, it is more probable that the bulk of the processing time is linked to the number of detected elements. For the 333K points Kasepuhan data, the algorithm managed to detect, segment, and classify the objects in 25.1 s. This was done in 83.56 s for the 1.8M points St-Pierre dataset, also with eight detected structures. Conversely, the Valentino dataset that consists of almost 10 times more points than Kasepuhan was processed in a little over five minutes in order to detect 20 structures. However, the overall processing time is still faster by at least a factor of 2 when roughly compared to the time it takes to perform the same task manually, without taking into account the time required to identify and classify each cluster into the appropriate classes. A quick comparison was also performed between our results and the results presented in [36] which also used the Valentino in their experiments with the PointNet++ DL approach. As has been previously mentioned, M_HERACLES managed to yield a median precision value of 66.68%, recall value of 88.16%, and F1 score of 75.92%. In [36], Valentino was used as the test dataset after the authors' DL algorithm was trained using another dataset and was classified into four classes, including columns. For the columns' class, the authors cited a value of 49.10% in precision, 70.02% in recall, \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Object**} & \multicolumn{2}{c}{**Point Number**} & \multicolumn{2}{c}{**Misclassed**} & \multirow{2}{*}{**True Positive**} & \multirow{2}{*}{**\%Unclassed**} & \multirow{2}{*}{**\%P**} & \multirow{2}{*}{**\%R**} & \multirow{2}{*}{**\%F1**} \\ \cline{2-2} \cline{5-10}  & **Manual** & & & & & & & & & \\ \hline V01 & 35 370 & 46 666 & 15 594 & 4 298 & 31 072 & 12.15 & 66.58 & 87.85 & 75.75 \\ \hline V02 & 35 845 & 47 358 & 15 744 & 4 231 & 31 614 & 11.80 & 66.76 & 88.20 & 75.99 \\ \hline V03 & 39 169 & 51 853 & 17 333 & 4 649 & 34 520 & 11.87 & 66.57 & 88.13 & 75.85 \\ \hline V04 & 40 155 & 51 923 & 17 010 & 5 242 & 34 913 & 13.05 & 67.24 & 86.95 & 75.83 \\ \hline V05 & 38 288 & 52 623 & 17 575 & 3 240 & 35 048 & 8.46 & 66.60 & 91.54 & 77.10 \\ \hline V06 & 39 689 & 53 016 & 17 406 & 4 079 & 35 610 & 10.28 & 67.17 & 89.72 & 76.82 \\ \hline  & & & & & & & & & \\ \hline  & & & & & & & & & & \\ \hline  & & & & & & & & & \\ \hline \hline \end{tabular}
\end{table}