###### Abstract  
It is challenging to extract reliefs from ancient steles due to their rough surfaces, which contain relief-like noise such as dents and scratches. In this paper, we propose a method to segment relief region from 3D scanned ancient stele by exploiting local surface characteristics. For each surface point, four points that are apart from the reference point along the direction of the principal curvatures of the point are identified. The spin images of the reference point and the four relative points are concatenated to provide additional local surface information of the reference point. A random forest model is trained with the local surface features and, thereafter, used to classify 3D surface point as relief or non-relief. To effectively distinguish relief from the degraded surface region containing relief-like noise, the model is trained using three-class labels consisting of relief, background, and degraded surface region. The initial three-class result obtained from the model is refined using the \(k\)-nearest neighbors algorithm, and, finally, the degraded region is re-labeled to background region. Experimental results show that the proposed method performed better than the state-of-the-art, SVM-based method with a margin of 0.68%, 3.53%, 2.25%, and 2.36%, in accuracy, precision, F1 score, and SIRI, respectively. When compared with the height- and curvature-based methods, the proposed method outperforms these existing methods with accuracy, precision, F1 score, and SIRI gains of over 4%, 20%, 11%, and 12%, respectively. spin image; cross spin image; relief extraction; random forest; machine learning +
Footnote †: journal: Medical  
vision [5; 6; 7; 8; 9; 10]. The decorrelation stretch (DStretch), an image processing plug-in made by [PERSON] [5], was applied to enhance images of faint reliefs carved on rocks [11]. DStretch involves the application of the Karhunen-Loeve transformation to a 2D colored image of the surface of a rock to obtain a 3 \(\times\) 3 transformation matrix. The product of the obtained transformation matrix and the image that contains faint reliefs on rock produced a new image with enhanced reliefs. However, the performance of this functional software is not reliable in extracting reliefs when the surfaces of the monuments contain severely damaged regions. In another study, a feature extraction method, known as the Hough transform, was used for the representation of surface features by applying the feature curve approximation function to a family of curves [9]. This transformation was demonstrated to be effective in the recognition of decorative patterns from the surface of defective or severely degraded archaeological objects. Recently, [PERSON] et al. [10] extended the application of Hough transform for surface representation by defining new set of rules and conditions for selecting suitable curve approximation that guarantees reliable recognition of decorative surface points. Furthermore, new curve models were analyzed and added to the existing family of curves to increase the accuracy of identifying the characteristic curves. With the introduction of sophisticated 3D scanners, it is now realistic to extract reliefs from 3D scanned surfaces of stone monuments. The 3D scanned monument is represented as surface mesh with triangular faces. Several 3D data processing techniques can thereafter be applied to the 3D mesh to gain meaningful insights about the ancient monuments, such as changing the angle of views at various scales without optical illusions in the material pattern or discoloration [6; 7; 12; 13; 14; 15]. Scanning defects in the form of outliers, noise, ghost geometry, and holes are usually introduced to the 3D mesh surface during the scanning process. A set of tools based on point modeling techniques was presented to address these problems [16]. The point modeling techniques mainly involve data structures, i.e., the moving least square optimization and point relaxation functions.