dataesr/scientific-paragraphs-categorization huggingface dataset:
This section will address the multi-scalar point processing part, starting from the scale of a neighborhood up to that of architectural elements. As previously evoked, in this study, the multi-scalar approach is used to progressively segment the point cloud of a heritage complex-first into building units and then further into architectural elements (e. g. , wooden frames and structural supports). The general flow of the approach can be seen in Figure  3. In addition to progressive segmentation, the developed method will also try to classify the results as automatically as possible in order to add the semantic dimension to the data, which is vital in BIM and 3D GIS environments. In this regard, a toolbox was created in the Matlab c environment to host all the codes and functions written for the study under one project: HERitAge by point CLoud procESsing for Matlab c (M_HERACLES).
The aim of M_HERACLES is to develop simple algorithms to help in the automation effort of point cloud processing in the context of cultural heritage. This includes among others segmentation, semantic annotation, and 3D primitive generation. The toolbox is open source and available online via GitHub (see Supplementary Materials for download link). M_HERACLES is developed in Matlab c R2018a using its Computer Vision Toolbox and several other third party libraries. The datasets were all processed using functions available in the M_HERACLES toolbox; the two main datasets (Kasepuhan and St-Pierre) were processed firstly on the scale level of a heritage complex to that of individual buildings (step 1 in Figure  3 ). The resulting sub-clouds were thereafter used to be further segmented on the building to architectural element scale level (step 2 in Figure  3 ).
Following the data integration workflow previously established via georeferencing to a common geodetic system, this prerequisite does not pose a problem. The algorithm starts by classifying the point cloud between ground and non-ground elements. The Cloth Simulation Filtering (CSF) method  [66]  was used in this regard. Algorithm 1 displays the pseudocode of the proposed segmentation algorithm used at the aftermath of the ground extraction process, as written in the function shapeseg. m. In essence, the function creates a 2. 5D bounding box from the geometry stored in the shapefile data. All point clouds of all altitudes located inside this "cookie cutter"-like bounding box were segmented into a single cluster. From this cluster, a subsequent Euclidean distance-based region growing was performed to separate the main object of interest from any possible noise, including those present due to the overstacking of vertical objects (e.
Results and Discussion The GIS shapefile data shown in Figure  4  were used to aid the segmentation process. In the case of the St-Pierre dataset, the shapefile was acquired through the open data framework of the Strasbourg municipal council, the "Référentiel topographiques simplifié" (RTS) or simplified topographic reference. The RTS shapefile data consist of several classes, but, for the purposes of this study, only the "public building" class will be addressed. For the Kasepuhan dataset, however, no prior shapefile was available for the site. The shapefiles of several object classes were therefore generated via digitization of the orthophoto of the site, which was also made available during the acquisition mission. The digitization was made to be not very precise on purpose, in order to test the robustness of the developed function.
The original Kasepuhan dataset consisted of 10. 4 million points, and were segmented into four classes (buildings, walls, gates, and the ground) and 13 different annotated objects in about 10 min. In the case of the St-Pierre dataset, the algorithm was visually more successful in segmenting the ALS point cloud into the public building class and annotating them. The St-Pierre dataset consisted of 5. 9 million points and was segmented into 17 objects of one class in about 7 min. Here, again, the visually higher success of the St-Pierre dataset may be due to the better CSF results, thus giving a cleaner result than Kasepuhan. Figure  5  shows the results for six of the most important and thus interesting heritage buildings within the class. Table  1  presents a quantitative analysis on the obtained results for the two datasets.
For the St-Pierre dataset, quantitative values seem to validate the qualitative visual inspection of Figure  5  that the algorithm worked better than for Kasepuhan. In the statistical analysis presented in Table  2 , only six of the most important heritage sites located within the neighborhood of the St-Pierre church were taken into account. In this reduced sample, the median unclassified percentage amounts to 6. 64% (i. e. , comparable to that of Kasepuhan), but the median precision attained a value of 98. 86% and the median recall that of 93. 69%, thus yielding a median F1 score of 93. 90%. The best result was obtained for the Palais des Fêtes (PLSFETES) building, with 100% precision (97. 5% F1 score). The worst F1 score was obtained by the Palais de Justice (PLSJUSTICE) building. This is due to a chunk of the point cloud of the said building which was visibly not segmented into the cluster.
This unsegmented chunk corresponds to the scaffoldings on the building, erected due to renovations. The ALS points of the scaffoldings were so few that M_HERACLES considered them as noise. Apart from this outlier, the most frequently encountered error seems to be related to the presence of vegetation or, analogous to the Kasepuhan data, the minor errors due to prior ground extraction. An interesting point to note in summarizing these results is the speed of the processing when compared to manual segmentation and labeling. The algorithm, while having several outliers (especially in the presence of important noise), generated good results. This is particularly true in the case of the St-Pierre dataset, where the urban density and particularly flat terrain generated very good results.
Comparison with a Commercial Solution In order to assess the quality of our developed approach, a comparison was performed with the automatic point cloud classification results of the commercial software Agisoft Metashape. While Metashape is chiefly a photogrammetric software known for its use in image-based reconstruction, it was also recently augmented with a function for multi-class point cloud semantic segmentation. According to the official documentation, Metashape employs a machine learning technique to perform this task; indeed, it asks its users to submit training datasets in order to improve the classification quality in the future. For the purposes of our comparison, Metashape version 1. 5. 3 build 8469 (release date 24 June 2019) was used. The Metashape automatic classification was performed on both main datasets: Kasepuhan and St-Pierre.
Visual results for the Kasepuhan can be seen in Figure  6. Three classes were defined, namely the 'buildings', 'walls' and 'trees' classes. In Metashape, this corresponds respectively to 'buildings', 'man-made objects', and 'high vegetation' classes. Figure  6  shows that the Metashape automatic classification had difficulties in distinguishing between buildings and walls, with most of the walls classified as buildings. Some parts of the walls were also classified as high vegetation. This may be due to the fact that the Kasepuhan dataset presented a large-scale and thus more complex scene not entirely suitable for the machine learning-trained function. Unfortunately there is no way to verify this hypothesis since Metashape understandably does not divulge their machine learning method in detail.
On the contrary, M_HERACLES managed to classify the objects fairly well thanks to the use of shapefiles to guide it. Visually, Figure  6c  also showed that some of parts were nevertheless unclassified, notably the walls at the back of the dataset. This may be due to the low resolution of the point cloud for this part of the site (note the same observation on Metashape results). Table  3  displays a quantitative comparison of the two tested algorithms for Kasepuhan, also visualized via histograms in Figure  7. M_HERACLES managed to outperform Metashape in most cases (yielding a slightly lower F1 score in the trees class), especially in the walls class. The median value of the F1 score for M_HERACLES was 85. 30% compared to Metashape's 60. 52%. It showed a lower recall value and higher precision, which may be explained by the fact that the use of shapefiles disproportionately increased Type I error.
When implemented on the St-Pierre dataset, M_HERACLES notably still performed better than Metashape as can be consulted in Figure  8. Metashape produced a very high precision rate; however, this must be understood with a caveat. Indeed, Metashape performed automatic segmentation on all buildings on the scene, whereas M_HERACLES only performed one on the "public building" class as dictated by the related shapefile. This distinction between public buildings and other buildings follows the official categorisation as set by the Strasbourg city geomatics service. In this regard, the results of Metashape were therefore manually segmented to include only the so-called public buildings, thus yielding a slight bias towards higher precision. However, as far as the recall value is concerned, M_HERACLES again outperformed Metashape.
• A final distance-based region growing was performed to eliminate any remaining noise. Thus, the output of the function is a structure of point cloud clusters, labeled as columns or non-columns. The inverse is seen in the Valentino dataset, where the non-planar ceiling created a case of oversegmentation. Indeed, the statistics as for Valentino show a high rate of recall but lower precision. The visual results for the St-Pierre dataset had been amply described in  [67] , including the overclassification of the iron fence attached to the posterior pillar. Statistically speaking, within Tables  4 5 6 , the overclassified column describes the number of points considered as false positives, while the unclassified column denotes the true negative False negative points are not shown since the values are negligible due to the cookie-cutter approach of taking all points of all elevations of a particular polygon shape.
Similar to the analysis conducted in Section 4. 2, four statistical values were presented to assess the quality of the algorithm, namely percentage of unclassified points, precision, recall, and F1 score. In terms of the unclassed percentage, Kasepuhan showed a higher rate (median of 34. 36%) which is most probably caused by the same reasons as the one established before regarding errors during the pre-segmentation between the body and the attic. The median precision of Kasepuhan is 100%, which is very satisfactory. However, as has been previously mentioned, its recall value is lower at 65. 64%. This loss in recall value also seems to be systematic, again validating the points as argued in the previous paragraph. The overall median F1 score for the Kasepuhan dataset was 79. 23%. The statistics for the St-Pierre dataset displayed a similar trend to that of the Kasepuhan; that is, it registered higher precision than a recall rate.
With a median precision of 81. 20% and recall of 71. 39%, the results for this dataset are nevertheless quite promising. It should well be noted that the St-Pierre choir dataset is quite complex due to the existence of the twin pillars and the presence of many noises (folded chairs were placed against the twin pillars in addition to the presence of the iron fence on the posterior pillar). Indeed, manual segmentation and labeling took quite some time to perform the task due to these conditions. Granted, the automatic results still had remaining noises and must be cleaned further manually. However, with a fast processing time (a little under one and a half minute), this solution may prove to be very useful in performing the segmentation task, or at least provide a first approximate result. Table  6.
Quantitative analysis on the results of step 2 for the detection and classification of columns in the Valentino dataset. Note that only the detected columns were taken into account here. %P is precision, %R is recall, and %F1 is the normalized F1 score. For the Valentino dataset, the unclassed rate stands at a median value of 11. 84%. The precision level is low for the Valentino, at 66. 68%, which suggested overclassification. As has been mentioned before, this is mainly due to the ceilings of the dataset that represent arcs instead of planar surfaces as had been hard-coded in the algorithm. A further improvement of the algorithm may incorporate this possibility into account, as this type of ceiling can be found in many heritage datasets. The recall value is, however, quite high, with a median value of 88.
Object As far as the classification goes, Figure  10  shows that the algorithm managed to correctly classify the pillars of the Kasepuhan dataset as non-columns. Indeed, under the definition of columns as set in this paper and contrary to classical columns, the eight structures in the Kasepuhan dataset cannot be classified as columns as they are in fact rectangular shaped posts. For the St-Pierre, the algorithm correctly determined that the eight detected structures are inside the "column" class. In the case of the Valentino, it also managed to correctly identify that the free-standing pillars are columns, while the rest of the detected structures were classified as non-columns. The processing time of the datasets showed that they may be, at least in part, linked to the number of points inside the input data.
However, it is more probable that the bulk of the processing time is linked to the number of detected elements. For the 333K points Kasepuhan data, the algorithm managed to detect, segment, and classify the objects in 25. 1 s. This was done in 83. 56 s for the 1. 8M points St-Pierre dataset, also with eight detected structures. Conversely, the Valentino dataset that consists of almost 10 times more points than Kasepuhan was processed in a little over five minutes in order to detect 20 structures. However, the overall processing time is still faster by at least a factor of 2 when roughly compared to the time it takes to perform the same task manually, without taking into account the time required to identify and classify each cluster into the appropriate classes. A quick comparison was also performed between our results and the results presented in  [36]  which also used the Valentino in their experiments with the PointNet++ DL approach.
As has been previously mentioned, M_HERACLES managed to yield a median precision value of 66. 68%, recall value of 88. 16%, and F1 score of 75. 92%. In  [36] , Valentino was used as the test dataset after the authors' DL algorithm was trained using another dataset and was classified into four classes, including columns. For the columns' class, the authors cited a value of 49. 10% in precision, 70. 02% in recall, and 57. 60% in F1 score (Figure  11 ). Although our algorithm managed to provide better results than the compared study, several remarks should nevertheless be taken into account. Firstly, in our study, only free-standing pillars were accounted for, whereas  [36]  also included engaged columns. Indeed, M_HERACLES did not manage to correctly detect the engaged columns. Secondly, the DL approach used in the other study has the potential to generate better results with more training data.
The idea behind the developed function is to first decompose the beam point clouds into facets. Afterwards, several geometric constraints were applied to extract the point cloud of individual beams from those of the facets. The functions were created to reach this point of the segmentation process. However, an optional third party library also enables the creating of parametric best-fit cuboids from the segmented beams. The overall workflow of the developed approach is described in Figure  12. The facet detection was performed using the region growing method. The theoretical primer of the region growing method is well known, and, in this case, we used the same approach as the Point Cloud Library (PCL)  [73]  but implemented it in Matlab c. This implementation employs point cloud normals and curvatures as constraints, as opposed to the function pcsegdist in Matlab c which uses Euclidean distance as the principal constraint.
An optional further processing involves using the RANSAC algorithm to generate a best-fit cuboid for each beam cluster. This was however done by a third-party Matlab c library (https://fr. mathworks. com/matlabcentral/fileexchange/65168-cuboid-fit-ransac, retrieved 28 January 2020). This part of the algorithm is still under development; however, a preliminary result (as shown in Table  7 ) conducted on a subset of the HK-Castle dataset showed that the algorithm managed to correctly identify the individual beams. The small dataset consists of 100k points and was processed in 3 min 42 s. The algorithm gave very good results in terms of precision (median value of 94. 45%), but quite low values of recall (median value of 75. 58%). The low recall value can be explained by the fact that the algorithm also performs noise reduction, in which detected regions having less than a set threshold number of points are eliminated.
The resulting cluster is therefore cleaner than the manual segmentation, but this means a sharp decrease in recall value. The precision value is however very satisfactory. Furthermore, the algorithm correctly deduced the number of beams that are present in the input point cloud. However, the algorithm still suffers in terms of processing time. More than half of the processing time was taken by the curvature computation at the beginning of the function; this requires therefore more investigation and optimization. Furthermore, these preliminary results concern only a small dataset. More investigations must be conducted to assess the quality of the algorithm, namely by processing a larger dataset.
That being said, the attained median F1 scores of 91. 99% for Kasepuhan and 93. 90% for St-Pierre are very encouraging. Comparison with an external solution (Agisoft Metashape) was also performed in this section. Results on both the Kasepuhan and St-Pierre datasets showed that M_HERACLES managed to outperform Metashape as regards the precision and quality of the classification process. However, it should be noted that Metashape uses a machine learning approach in which the availability of training data are paramount. We are quite confident that machine learning solutions will become better as time goes; indeed, one of the main objectives of M_HERACLES is not to confront its performance directly with machine learning solutions, but rather to complement it via e. g. , automation of training data creation.
In Section 4. 2, tests on two additional datasets in addition to the results described in  [67]  showed that the algorithm is useful in performing fast segmentation and classification for structural supports. However, as has been shown in the results, the algorithm, while fast and easy to use, remains prone to noise and deviations from the hard-coded geometrical rules. This is evidenced by the stark contrast between the three datasets, whereas Kasepuhan presented cases with higher precision but lower recall (suggesting underclassification), the Valentino showing higher recall but lower precision (suggesting overclassification) and the St-Pierre presenting somewhat of a mix between the two cases. These differences were caused by deviations from the general rule; the Kasepuhan interior ceiling does not correspond to the altitude of the roofs on the exterior, while Valentino's non-planar ceiling caused the error.
The St-Pierre case showed that the proneness of the algorithm against noises. However, the results remain promising and some lessons can be learned from this experiment, to be the subject of further improvement. A comparison with the DL approach presented in  [36]  for the Valentino also showed a rather favorable outcome with respect to the quality parameters. Furthermore, the fast nature of the segmentation and classification process is in contrast to training-intensive ML/DL and resource-intensive manual segmentation and labeling. It may therefore be used to complement ML/DL algorithms, especially in the generation of training data. The beam detection algorithm described in Section 4. 3 is still in its early stages, and more tests must be conducted in order to assess its efficacy. A test with a small dataset yielded a very satisfactory median precision value of 94.
Results and Analysis A substantial test regarding the results of this section was presented in  [67]. In this paper, two new datasets were introduced: the Central Pavilion of the Kasepuhan dataset and the Valentino dataset, while the St-Pierre dataset was presented with new statistics. The results can be consulted in Figure  10. The Kasepuhan dataset is small in size, with a little over 333K points while the St-Pierre and Valentino datasets presented a much larger point cloud with over 1. 8M and 3. 5M points, respectively. It is also interesting to note that the three tested datasets possess different styles of architecture; the Kasepuhan dataset consists mainly of open pavilions with many free-standing columns while the Valentino presents an example of an interior point cloud case. The St-Pierre church choir was chosen due to its particularity in that it possesses twin pillars instead of the usual free-standing ones.
As can be seen in Figure  10 , in terms of the segmentation process, the algorithm managed to detect eight structural supports in the Kasepuhan dataset. This corresponds exactly with the ground truth data. In the case of the St-Pierre, it managed to correctly detect the eight pillars individually despite the twin pillar nature. For the Valentino, 20 structures were detected in lieu of the actual 19 as can be found in the ground truth. The Valentino data presented a particular challenge since 13 out of the 19 pillars present in the dataset are in the form of engaged pillars, i. e. , semi-pillars or columns which are part of the wall. As can be seen from the results, the algorithm had difficulties in segmenting these kinds of structural supports, while having no problem for free-standing pillars.
For the Kasepuhan dataset, a preliminary segmentation was performed to divide it into the building body and attic, with the body used as input for the function supportdetect. m as described in Algorithm 2. Some points mainly at the top of the pillars remained unclassified. This is due to the fact that, in the pre-segmentation of the building body and attic, the algorithm considered the change in the surface of the cross-sections of the building to determine the two parts. These cross-section surfaces being calculated from the surface of the bounding-box, only the exterior point cloud was considered instead of the interior. This is reflected numerically in Table  4 , where the recall value for this dataset is visibly low despite a very high precision.
Supplementary Materials: M_HERACLES is an open source toolbox written in Matlab c. The toolbox with all the functions described in this paper can be downloaded via GitHub in the following webpage: https://github. com/murtiad/M_HERACLES (last updated 20 January 2020). Author Contributions: Conceptualization, A. M. and P. G. ; software, A. M. ; writing-original draft preparation, A. M. ; writing-review and editing, P. G. ; supervision, P. G. All authors have read and agreed to the published version of the manuscript.
Funding: The research is part of a PhD project funded by the Indonesian Endowment Fund for Education (LPDP), Republic of Indonesia. The Kasepuhan dataset was also acquired through the Franco-Indonesian Partenariat Hubert-Curien (PHC) NUSANTARA program under the aegis of the Indonesian Ministry of Research and Higher Education (KEMENRISTEKDIKTI), the French Ministry for Europe and Foreign Affairs (MEAE), and the French Ministry of Higher Education, Research, and Innovation (MESRI).