dataesr/scientific-paragraphs-categorization huggingface dataset:
The two steps of data segmentation and H-BIM reconstruction are both completed by a data validation process. survey. The workflow for this first phase was previously presented in reference  [76] , to which the reader could refer for further details and in-depth discussion on the ML-based data processing step. The focus of this paper is rather on the second part of the workflow, concerning Scanto-BIM reconstruction based on semantically annotated data. The latter takes place following the successive steps of: import of annotated data into the BIM environment; reconstruction and propagation, via VPL, of template geometries based on knowledge derived from historical treatises of architecture; and transfer of template geometries into BIM software as Autodesk Revit (Figure  6 ). The two steps of data segmentation and H-BIM reconstruction are both completed by a data validation process.
Semantic Segmentation via ML At first, semi-automated systems were exploited to properly interpret the 3D architectural scene from the input point cloud to improve the description and recognition of forms, materials, state of preservation. A ML-based segmentation procedure was used to assist the proper processing, management and semantic enrichment of digital heritage objects. The RF by Breiman  [74]  was used as the reference algorithm following the successful tests by  Grilli [72]. This classifier is recognized as an effective tool for the classification of typological components of a building, outperforming other ML and DL approaches in terms of trade-off between training time, size of training data required and accuracy of the obtained results  [26]. Starting from the extraction of appropriate covariance and radiometric features, and using a relevant set of training data, the RF is trained to classify, within digital models, the architectural elements that make up a historic building  [76].
Semantic Segmentation via ML At first, semi-automated systems were exploited to properly interpret the 3D architectural scene from the input point cloud to improve the description and recognition of forms, materials, state of preservation. A ML-based segmentation procedure was used to assist the proper processing, management and semantic enrichment of digital heritage objects. The RF by Breiman  [74]  was used as the reference algorithm following the successful tests by Grilli  [72]. This classifier is recognized as an effective tool for the classification of typological components of a building, outperforming other ML and DL approaches in terms of trade-off between training time, size of training data required and accuracy of the obtained results  [26]. Starting from the extraction of appropriate covariance and radiometric features, and using a relevant set of training data, the RF is trained to classify, within digital models, the architectural elements that make up a historic building  [76].
For this reason, steps (ii) and (iii) are strictly interrelated. For the RF, the number of trees, N trees , is set to 100 and the hyperparameter optimization accounts for overfitting through a 10-fold cross validation procedure. Upon its completion of training using the training set of annotated data, the predictive model is applied to the remaining part of the point cloud or image (non-manually annotated), so to semantically label the classes of typological elements in the rest of the dataset. The accuracy of the classifier is finally assessed (iv), based on the comparison between true and predicted values on a validation set, that consists of almost the 25% of the labeled data (not used in the training phase). The performance evaluation is sorted out in the form of a confusion matrix, providing a measure of the number of correct and incorrect predictions, class by class.
The on-diagonal elements stand for the True Positive (TP) values (correctly classified instances of the dataset), while the off-diagonal elements provide a measure of misclassifications: True Negatives (TN), False Positives (FP) and False Negatives (FN) values. The performance measures of Precision, Recall, Overall Accuracy and F-measure are derived from a combination of these values, as follows: Precision = TP TP + FP (1) Recall = TP TP + FN (2) Overall accuracy = TP + TN TP + TN + FP + FN F -measure = 2• Recall •Precision Recall + Precision (3) The point cloud obtained at the end of this classification step is separated into its recurring architectural components: considering that the semantic structuring of heritage data could be beneficial in view of the construction of BIM-based representations, this classified point cloud is taken as initial data for the Scan-to-BIM reconstruction process.
This way of classifying data appears consistent with the logic of the H-BIM process, whereby the model results from a combination of smart objects, properly differentiated in terms of type and morphology (e. g. , roof, wall, floor, column, etc. ) and grouped into families of architectural elements. In the overall model, each component is effectively discerned according to whether it belongs to one class or another. The model obtained at the end of the process, containing the 3D reconstruction of all typological element classes, can be used to construct H-BIM type representations, i. e. , to build 3D archives of architectural heritage, which can be further enriched with information related to preservation and documentation. In detail, for the reconstruction of template forms, reference geometries and proportions are derived, where available, from historical architectural treatises.
e. , definition of element replica operations, so to enable the duplication of the defined geometry to multiple elements sharing same characteristics. The mathematical and conceptual representation of each class, managed through generative modeling procedures, is entrusted to the creation of Non-Uniform Rational B-Splines (NURBS). Real-time generation, control, and editing of architectural forms is accomplished through the Grasshopper graphical algorithm editor, integrated in Rhino McNeel. Grasshopper, in particular, allows visual control of the 3D geometries reconstruction procedures by direct manipulation of nodes (algorithms) and wires. Finally, the Volvox (food4rhino. com/ en/app/volvox, accessed on 18 December 2022) and Rhino. Inside. Revit (rhino3d. com/ inside/revit/1. 0/, accessed on 18 December 2022) plug-ins are used to connect the reconstructed 3D model with point cloud processing software and BIM platforms, respectively.
Annotated 3D Data At an initial stage, geometric features are extracted from input raw point clouds, with varying local neighborhood radii for each 3D point. Features, as covariance characteristics and changes of curvature, are computed and iteratively selected for each case study via a predictor importance estimate process  [76]  (Figure  7 ) in order to allow better distinction between one class and another. The choice of the local neighborhood in which features vary draws on considerations of the recurring dimensions of many elements composing the dataset, that are provided with a first estimate (e. g. , the diameter of the columns, or thickness of certain architectural moldings, as well as other repetitive dimensions of the elements) so to extract geometric features considering selected ranges of the local neighborhood (e.
g. , ρ = 0. 2 m; ρ = 0. 4 m; ρ = 0. 6 m). Within the chosen ranges of ρ, the covariance features are extracted from the covariance matrix, while the Normal Change Rate is extracted as a curvature measure describing for each point the speed of the orientation change  [38]. Radiometric features, derived from the decomposition of the color space into single R, G and B scalar fields, and a height feature (the Z coordinate), are then considered as additional characteristics for each dataset. For additional samples of extracted covariance features -i. e. , features depending on the distribution of 3D points in space (linearity, planarity, sphericity, omnivariance, etc. ), colorimetric features and height features, the readers can refer to Appendix A (Figures  A1  and A2 ). After feature extraction, classes of architectural components are identified and annotated on a reduced portion of data samples, consisting of almost 20% of the total number of points of the point cloud (Figure  8 ).
The so-called training set is specified each time for each case study; the point cloud is segmented based on the ten benchmark classes proposed by Matrone et al. , 2020  [77]  (Figure  9 ): arch, column, molding, floor, openings (door or window), wall, stair, vault, roof and other (all elements not belonging to the previous classes). The overall number of cases is 10 for the Grand-ducal cloister dataset, but it was reduced to 9 for the Grand cloister, as there is no 'Class 6-Stair', as well as for the San Matteo dataset, where 'Class 8-Roof' was not visible since the photogrammetric survey was ground-based and did not allow the description of the roofing structure. In any case, each identified class, for the three datasets, was associated with a specific label, to a class index varying from 0 to 10 and to a related color; the training set was chosen in a representative portion of the dataset, where all classes to be annotated are visible.
duced to 9 for the Grand cloister, as there is no 'Class 6-Stair', as well as for the San Matteo dataset, where 'Class 8-Roof' was not visible since the photogrammetric survey was ground-based and did not allow the description of the roofing structure. In any case, each identified class, for the three datasets, was associated with a specific label, to a class index varying from 0 to 10 and to a related color; the training set was chosen in a representative portion of the dataset, where all classes to be annotated are visible. The overall number of cases is 10 for the Grand-ducal cloister dataset, but it was reduced to 9 for the Grand cloister, as there is no 'Class 6-Stair', as well as for the San Matteo dataset, where 'Class 8-Roof' was not visible since the photogrammetric survey was ground-based and did not allow the description of the roofing structure.
In any case, each identified class, for the three datasets, was associated with a specific label, to a class index varying from 0 to 10 and to a related color; the training set was chosen in a representative portion of the dataset, where all classes to be annotated are visible. After the manual annotation of the training set, a first classification is run, via the RF algorithm, considering the multi-scale feature extraction (i. e. , the whole set of features extracted at different local neighborhood radii). At the end of the learning process, an importance ranking can be displayed, showing the relevance score of each feature. Starting from the importance ranking extracted, redundant and less relevant features are iteratively removed, and the RF is hence trained with a reduced subset of features.
The feature ranking process is run each time for the three different case studies; surface variation, sphericity, anisotropy and verticality always appear among the most relevant features. By comparing the results of the three different datasets in terms of feature selection, and by visually considering the variation of features along the datasets, many relevant considerations can be drawn up on the recurrence of some features: verticality, for instance, is more apt to the distinction between elements of the dataset that are mostly horizontal (floors, ground) or vertical (columns, walls), while omnivariance and sphericity By comparing the results of the three different datasets in terms of feature selection, and by visually considering the variation of features along the datasets, many relevant considerations can be drawn up on the recurrence of some features: verticality, for instance, is more apt to the distinction between elements of the dataset that are mostly horizontal (floors, ground) or vertical (columns, walls), while omnivariance and sphericity enable the recognition of architectural moldings, arches, vaults and columns; anisotropy and planarity further support the classification of columns and the distinction of windows and doors from the wall.
The original point cloud is segmented into multiple point clouds, each containing the points belonging to the individual class of architectural elements (2); then, the management of a special slider allows the user to directly select the index of a desired class (3). The colors, names and points associated with the selected class are selected (4) and the corresponding point cloud is displayed in the graphical user interface  (5). The advantage of this procedure lies in allowing the user to activate or deactivate the display of a given class, depending on case-specific needs: only the 3D points belonging to the selected class are visible, while the remaining 3D points are hidden (Figure  14 ). This direct manipulation of architectural component classes on the original point cloud simplifies the modeling phase that is already in its first stage since the 3D reconstruction work can be carried out on an already segmented (reduced) dataset.
Libraries of Template Geometries Once the import of the semantically segmented point cloud into the modeling environment is completed, each class can be reconstructed in the form of a template geometry, i. e. , a parametric 3D object that reproduces a particular recurring typological element (class or family of parametric elements). Again, the need to reconstruct geometries with modifiable and adaptable parameters leads to the choice of VPL, to model and edit the model by means of a series of rules and graphical processing operations (e. g. , through the connection of nodes and wires in the VPL of Rhino Grasshopper). The following operations are performed via VPL algorithms:  This direct manipulation of architectural component classes on the original point cloud simplifies the modeling phase that is already in its first stage since the 3D reconstruction work can be carried out on an already segmented (reduced) dataset.
Libraries of Template Geometries Once the import of the semantically segmented point cloud into the modeling environment is completed, each class can be reconstructed in the form of a template geometry, i. e. , a parametric 3D object that reproduces a particular recurring typological element (class or family of parametric elements). Again, the need to reconstruct geometries with modifiable and adaptable parameters leads to the choice of VPL, to model and edit the model by means of a series of rules and graphical processing operations (e. g. , through the This direct manipulation of architectural component classes on the original point cloud simplifies the modeling phase that is already in its first stage since the 3D reconstruction work can be carried out on an already segmented (reduced) dataset.
e. , a deformable section that defines the height and width of the element. A starting point and an ending point are the anchor points for any other moldings attached to the element. Considering the insertion points of the moldings as anchoring elements for the construction of subsequent moldings, more complex profiles are later provided (Figure  17 ). The application of extrusion, loft, sweep and revolution functions to the constructed generating profiles determines the template shape of a class. Figure  18  shows the construction process of the 'column shaft class', for the case of the Grand Cloister dataset: first, descriptors and geometric attributes are defined for the construction of this architectural component. Then, the column shaft is built based on the study of the dimensional relationships between the diameter of the column base and its height, which allows for the establishment, at different heights, of reference circles.
Information Propagation and Import into BIM Software Once defined for each class, the geometry of the model is subsequently propagated to other parts of the point cloud, where the 3D points have been recognized as belonging to the same architectural type. The procedure is done through duplication and displacement nodes, particularly leveraging array, copy and translation operations (e. g. , the Move node in Grasshopper). These operations allow the repetition, as many times as necessary, of the 3D geometry for each class of architectural components (Figure  19 ). Figure  20  shows the results obtained in the construction of some significant classes extracted from the considered datasets: the 'column' class for the main cloister, the 'vault' class for the Grand-Ducal cloister and the 'arch' class for the Museum of San Matteo.
Assessment of ML-Based Classification Methods Upon assessment of the performance scores of the three datasets being considered, the geometry-based classification returned an average accuracy of 98. 73% and an F-score of 87. 13%. The models accurately predicted the semantic segmentation results and significantly reduced the annotation time: Figure  24  summarizes the processing times for different stages of the workflow. However, some considerations need to be raised on the validation of this methodology: Following these principles, each element is reconstructed independently of the software used and can be imported, for example, into BIM software, to be further enriched with non-geometric information (related, e. g. , to materials, restoration and consolidation work, documentary and analytical sources, state of preservation, etc.
). For instance, Grasshopper VPL could be linked to Autodesk Revit BIM software via the Rhino. Inside. Revit plug-in, as one possible way of importing the classified model in BIM environment: the algorithm displayed in Figure  23   Following these principles, each element is reconstructed independently of the software used and can be imported, for example, into BIM software, to be further enriched with non-geometric information (related, e. g. , to materials, restoration and consolidation work, documentary and analytical sources, state of preservation, etc. ). For instance, Grasshopper VPL could be linked to Autodesk Revit BIM software via the Rhino. Inside. Revit plug-in, as one possible way of importing the classified model in BIM environment: the algorithm displayed in Figure  23  allows the selection of architectural objects belonging to individual element classes and associates them with a Revit family, preserving their level of semantic description.
This also implies that Grasshopper and Rhino3D do not represent the only environment in which such methods could be implemented, but rather other generative modeling software (such as Dynamo, implemented in Autodesk Revit software) could be exploited for the same task, if the same construction, modeling and operation processes are retrieved and repurposed accordingly. In addition, as a result of the connection drawn between the reconstructed classes and the semantic point cloud, it is pertinent to note that the two types of representations could be compared with each other, e. g. , in terms of relative distances, in order to derive the extent to which, for each class of elements, a quasi-conceptual (digitally reconstructed) model deviates from the real data (e. g. , from a point cloud acquired by survey).
In addition, high performance computing machines are required in the feature extraction and datadriven algorithm training phase. Moreover, mis-classification may occur in boundary regions, as well as in regions with similar development (geometry-based approaches) or with similar color characteristics or patterns (texture-based approaches). The generalization of the same ML algorithm to other datasets, pertaining to different architectural types and/or built in different periods, as well as the establishment of larger annotated datasets to train deep neural networks, are possible future developments in this domain. The semantically segmented point cloud is later exploited for the construction of a reference model, composed of template geometries, following the logic of H-BIM type information systems.
In addition, high performance computing machines are required in the feature extraction and data-driven algorithm training phase. Moreover, mis-classification may occur in boundary regions, as well as in regions with similar development (geometry-based approaches) or with similar color characteristics or patterns (texture-based approaches). The generalization of the same ML algorithm to other datasets, pertaining to different architectural types and/or built in different periods, as well as the establishment of larger annotated datasets to train deep neural networks, are possible future developments in this domain. The semantically segmented point cloud is later exploited for the construction of a reference model, composed of template geometries, following the logic of H-BIM type information systems.
The feature ranking process is run each time for the three different case studies; surface variation, sphericity, anisotropy and verticality always appear among the most relevant features. By comparing the results of the three different datasets in terms of feature selection, and by visually considering the variation of features along the datasets, many relevant considerations can be drawn up on the recurrence of some features: verticality, for instance, is more apt to the distinction between elements of the dataset that are mostly horizontal (floors, ground) or vertical (columns, walls), while omnivariance and sphericity.
6 m) values of ρ. • Regions with similar developments (e. g. , planar or cylindrical) in which geometric features may yield several values, can be misclassified as falling into the same class. For instance, the analysis of the off-diagonal elements of the confusion matrices suggested that Class 5-Wall' and 'Class 4-Door and Window' are often interchanged with each other, as both are characterized by predominantly planar behaviors. • As the covariance and curvature features are computed in a given local neighborhood, the density of the point cloud influences the classification results. In other words, if two point clouds of a same object have different point densities, feature selection may produce two different results, as seen in the example in Figure 26. In order to align feature selection for different surveys of the same dataset, one could then plan to return the point clouds to the same density by means of a subsampling operation.
Regions with similar developments (e. g. , planar or cylindrical) in which geometric features may yield several values, can be misclassified as falling into the same class. For instance, the analysis of the off-diagonal elements of the confusion matrices suggested that Class 5-Wall' and 'Class 4-Door and Window' are often interchanged with each other, as both are characterized by predominantly planar behaviors. •As the covariance and curvature features are computed in a given local neighborhood, the density of the point cloud influences the classification results. In other words, if two point clouds of a same object have different point densities, feature selection may produce two different results, as seen in the example in Figure26. In order to align feature selection for different surveys of the same dataset, one could then plan to return the point clouds to the same density by means of a subsampling operation.
• As the covariance and curvature features are computed in a given local neighborhood, the density of the point cloud influences the classification results. In other words, if two point clouds of a same object have different point densities, feature selection may produce two different results, as seen in the example in Figure26. In order to align feature selection for different surveys of the same dataset, one could then plan to return the point clouds to the same density by means of a subsampling operation.