# A Preprocessing Framework for MeshCNN-based Surface Segmentation of Fragmented 3D Heritage Objects

Vinh-Nam Huynh, Hoang Ha Nguyen, Quoc Viet Kieu, Romain Raffin

<figure><img src="image_1.png" /></figure>

To cite this version:

Vinh-Nam Huynh, Hoang Ha Nguyen, Quoc Viet Kieu, Romain Raffin. A Preprocessing Framework for MeshCNN-based Surface Segmentation of Fragmented 3D Heritage Objects. VNICT2025: The 28th Vietnam Conference of Selected ICT Problems - VNICT2025, Nov 2025, Ninh Bình, Vietnam. pp.8534. hal-05373961

HAL Id: hal-05373961

https://hal.science/hal-05373961v1

Submitted on 20 Nov 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Preprocessing Framework for MeshCNN-based Surface Segmentation of Fragmented 3D Heritage Objects

Vinh Nam Huynh*, Hoang Ha Nguyen*, Quoc Viet Kieu*, Romain Raffin†

*ICT Department, University of Science and Technology of Hanoi,
Vietnam Academy of Science and Technology,

18, Hoang Quoc Viet, Nghia Do, Hanoi, Vietnam

†LIB, Université Bourgogne Europe,
9, av. A. Savary, Dijon, France

Email: {huynh-vinh.nam, nguyen-hoang.ha, kieu-quoc.viet}@usth.edu.vn, romain.raffin@u-bourgogne.fr

**Tóm tắt nội dung**—Surface segmentation of 3D meshes is critical for digital reconstruction and cultural heritage preservation. Deep learning approaches such as MeshCNN can learn directly from mesh geometry, but cannot process archaeological fragments due to incompatible input requirements. Fractured heritage meshes vary widely in resolution (2,380–111,788 faces) and contain non-manifold edges and topological defects, while MeshCNN requires fixed edge counts and manifold connectivity. We propose a preprocessing framework that addresses this problem through resolution normalization to 3,000 faces, topology validation, and label reprojection via surface distance mapping. Experiments on 36 archaeological fragments show that MS-DBSCAN achieves 67.93% region-based accuracy on preprocessed meshes, confirming that the framework preserves segmentation quality. MeshCNN achieves 44.40% accuracy, demonstrating that edge-based deep learning can now operate on heritage data. The performance gap reflects limited training data (27 fragments) rather than preprocessing failures. This work establishes preprocessing as necessary for applying deep learning to archaeological meshes and identifies few-shot learning as an important direction for future work.

**Keywords**-3D Mesh Segmentation; Cultural Heritage; Archaeological Fragments; Preprocessing Framework; MeshCNN; Few-shot Learning.

## I. INTRODUCTION

Conservation of cultural heritage relies on accurate digital representations of artifacts. Advances in 3D scanning now allow detailed replicas that support both analysis and preservation. To be useful, these 3D models must be segmented into meaningful surface regions that reflect different parts or physical conditions.

Segmenting archaeological meshes is difficult due to irregular topology, scanning noise, and varying resolution. Classical methods based on curvature or spectral cues often fail under these

conditions. Clustering algorithms such as DBSCAN variants have been adapted successfully to archaeological surfaces [1], but still depend on handcrafted features and sensitive parameter settings. Deep learning methods, by contrast, can learn directly from geometry. Among them, MeshCNN [2] has shown strong results on synthetic benchmarks, but requires standardized mesh structures rarely present in archaeological scans.

The core challenge is the incompatibility between archaeological mesh characteristics and deep learning requirements. Fractured heritage meshes from the TU Wien 3D Puzzles collection [3] range from 2,380 to 111,788 faces after Poisson reconstruction—a 47-fold variation that prevents direct MeshCNN application. Beyond resolution inconsistency, these meshes contain non-manifold edges, disconnected components, and holes that violate network assumptions. MeshCNN requires fixed edge counts across training samples to enable batch processing and edge-based convolution operations. No existing preprocessing pipeline addresses this adaptation systematically for heritage applications.

This paper introduces a preprocessing framework that makes MeshCNN feasible for archaeological surface segmentation. The framework includes resolution normalization to 3,000 faces, topology validation, and label reprojection via surface distance mapping, producing consistent per-edge labels for training. By standardizing inputs while preserving geometric detail, it enables edge-based deep learning to operate on fractured heritage data for the first time.

Our contributions are threefold:

* We propose a reproducible preprocessing framework that transforms irregular archaeological meshes into MeshCNN-compatible in-
puts through resolution normalization (Algorithm 1) and label reprojection (Algorithm 2).

* We demonstrate that preprocessing preserves segmentation quality: MS-DBSCAN achieves 67.93% region-based accuracy on preprocessed meshes, validating our methodology. MeshCNN achieves 44.40%, establishing the feasibility of deep learning on heritage data.

* We conduct an ablation study validating the 3,000-face resolution choice across material types, and identify few-shot learning as critical for future work given limited training data (27 fragments).

The remainder of this paper is organized as follows. Section II reviews related work on 3D mesh segmentation. Section III details the preprocessing framework and its integration with MeshCNN. Section IV presents the experimental setup, results, ablation study, and discussion. Finally, Section V concludes the paper and highlights directions for future research.

## II. RELATED WORKS

### A. Surface Segmentation in Cultural Heritage

Surface segmentation of 3D meshes is a long-studied problem in geometry processing [4]. Classical methods rely on curvature, dihedral angles, or spectral embeddings to detect boundaries[5], performing well on clean CAD or synthetic objects but struggling with heritage data due to erosion, scanning noise, and irregular sampling[6].

Surface segmentation differs from part segmentation [7], common in datasets like ShapeNet Part [8], which divides objects into functional components (chair legs, airplane wings). Part segmentation assumes consistent structures across objects. In contrast, heritage surface segmentation addresses irregular fragments divided into regions reflecting preservation state and breakage patterns. The lack of structural regularity and degraded geometry limit direct transfer of part segmentation methods, while multi-surface segmentation provides cues supporting fragment alignment and reassembly.

### B. Clustering-based Methods

To overcome the limitations of classical surface segmentation approaches on cultural heritage meshes, clustering methods avoid strong geometric assumptions. Algorithms such as k-means, spectral clustering, and DBSCAN group mesh elements based on similarity [9]. MS-DBSCAN [1]

extends this idea with multiple descriptors and has shown effectiveness on rough archaeological surfaces. These methods are flexible and can process arbitrary resolutions, but remain sensitive to parameters and depend on handcrafted features.

### C. Deep Learning on 3D Meshes

Although flexible, clustering methods depend on manually designed features, motivating deep learning approaches that learn representations directly from data [6]. Point-based models such as Point-Net [10] and PointNet++ treat geometry as unordered point sets, discarding connectivity information. Voxel CNNs [11] and graph-based approaches (GraphCNN, DGCNN) use different spatial representations than meshes.

MeshCNN [2] defines convolution and pooling directly on mesh edges, preserving topological relationships between adjacent faces—essential for surface segmentation where identifying coherent regions depends on understanding how neighboring faces relate. This makes MeshCNN methodologically comparable to MS-DBSCAN [1]: both exploit mesh connectivity and extract geometric features (normals, curvature, angles), differing primarily in whether features are handcrafted or learned. MeshCNN has performed well on synthetic benchmarks but has not been widely applied to cultural heritage due to incompatible input requirements that our preprocessing framework addresses.

## III. METHODOLOGY

### A. Challenges of Archaeological Meshes

Meshes from archaeological scans exhibit irregular topology, non-manifold edges, holes, and disconnected components—properties that hinder direct MeshCNN use, which requires consistent edge counts. Table I shows the dataset characteristics after Poisson reconstruction, illustrating the wide variability in fragment size and surface complexity. Our preprocessing framework addresses these challenges by standardizing topology, resolution (3,000 faces), and annotations, enabling MeshCNN to process heritage data while preserving surface detail necessary for segmentation.

### B. Preprocessing Pipeline

Our framework consists of four main stages:

1) **Ground truth creation:** Manual multi-surface segmentation in Meshlab guided by
<figure><img src="image_3.png" /><figcaption>Hình 1: Archaeological mesh preprocessing pipeline for MeshCNN adaptation showing parallel ground truth creation and remeshing.</figcaption></figure>

geometric cues. Detailed protocol in our previous work [1].

2) **Remeshing:** Normalization to 3,000 faces via Algorithm 1, ensuring consistent edge counts for MeshCNN.

3) **Cleaning and validation:** Removal of degenerate geometry and verification of watertightness and manifold connectivity.

4) **Label reprojection:** Algorithm 2 reprojects annotations via surface distance mapping to maintain ground truth consistency.

This preprocessing ensures consistent inputs while preserving geometric detail, providing a reproducible alternative to ad hoc remeshing. Figure 1 illustrates the complete workflow.

### C. MeshCNN Integration

After preprocessing (Algorithm 1), meshes adapt to MeshCNN's edge-based architecture [2]. Each edge encodes 5D features (dihedral angle, two inner angles, two edge-length ratios), invariant to translation, rotation, and uniform scaling. Edge-based convolutions predict per-edge class labels via cross-entropy loss.

### D. Resolution Choice

Resolution normalization is a critical component of our preprocessing framework. Algorithm 1 normalizes all meshes to 3,000 faces through edge collapse or subdivision. This choice balances three factors:

**Architectural compatibility.** For closed triangular manifolds, Euler's formula gives $E \approx 1.5F$, where *E* is the number of edges and *F* is the number of faces. Therefore, 3,000 faces yield approximately 4,500 edges, matching MeshCNN's input dimension of 4,600. Higher resolutions (5,000 faces → 7,500 edges) exceed GPU memory capacity (4GB) during batch training. Lower resolutions (2,000 faces) would require modifying MeshCNN's pooling sequence.

**Geometric detail preservation.** Preliminary experiments on 6 fragments showed that 2,000 faces loses critical surface details such as fine cracks and fracture boundaries, while 3,000 faces preserves features necessary for segmentation. Visual inspection confirms that boundaries remain identifiable after remeshing to 3,000 faces.

**Dataset accommodation.** Our dataset ranges from 2,380 to 111,788 faces after Poisson reconstruction. A target of 3,000 faces minimizes
**Algorithm 1:** Mesh Resolution Normalization

**Input:** Raw mesh $M_{raw}$, target

$$n_{target} = 3000$$

**Output:** Clean mesh $M_{clean}$ with $n_{target}$ faces

1. $n \leftarrow |\text{faces}(M_{raw})|$;

2. **if** $n > n_{target}$ **then**

3. $M \leftarrow \text{EdgeCollapse}(M_{raw}, n_{target})$;

4. **else**

5. **if** $n < n_{target}$ **then**

6. $M \leftarrow \text{LoopSubdivision}(M_{raw}, n_{target})$;

7. **else**

8. $M \leftarrow M_{raw}$;

9. Remove degenerate faces, fix normals, verify topology;

10. **return** $M_{clean} \leftarrow M$

information loss for smaller fragments (requiring upsampling from 2,380) while providing significant simplification for larger ones (downsampling from 111,788).

This design choice ensures the preprocessing framework can handle the full range of archaeological mesh variability while maintaining MeshCNN compatibility. Preliminary experiments on 6 fragments confirmed these design choices across material types (stone, clay, mortar).

## E. Label Projection and Edge Annotation

Ground truth annotations must be reprojected after remeshing to enable training with MeshCNN. Algorithm 2 computes edge labels through surface distance mapping to handle irregular archaeological boundaries.

The process assigns face centroids to ground truth segments based on minimum surface distance to handle irregular archaeological boundaries. Edge labels are determined by sampling five points along each edge and selecting the most frequent face assignment. This approach addresses boundary ambiguities where edges cross segment boundaries.

# IV. EXPERIMENTS AND RESULTS

## A. Dataset

We evaluate on the TU Wien 3D Puzzles collection [3]: fractured stone (Brick), clay (Venus,

**Algorithm 2:** Surface Distance Edge Annotation

**Input:** Mesh $M = (V, F)$, GT segments

$$\{S_1, \dots, S_k\}$$

**Output:** Edge labels $L_E$

1. $C_F \leftarrow \text{face centroids of } F$;

2. **for** $c_i \in C_F$ **do**

3. $L_F[i] \leftarrow \arg \min_j \text{ClosestSurface}(c_i, S_j)$;

4. $E \leftarrow \text{unique triangle edge pairs from } F$;

5. **for** $e_j = (v_a, v_b) \in E$ **do**

6. $P_j \leftarrow \{v_a + \frac{t}{4}(v_b - v_a) : t \in \{0, 1, 2, 3, 4\}\}$;

7. **for** $p \in P_j$ **do**

8. $f^* \leftarrow \arg \min_{f_i} \text{PointToTriangleDistance}(p, f_i)$;

9. $\text{label}(p) \leftarrow L_F[f^*]$;

10. $L_E[j] \leftarrow \text{mode}(\{\text{label}(p) : p \in P_j\})$;

11. **return** $L_E$

<table><caption>Bảng I: TU Wien Dataset: Simplified Mesh Characteristics (after Poisson reconstruction)</caption><thead><tr><td>Material Type</td><td>Fragment Count</td><td>Triangle Count</td></tr></thead><tbody><tr><td>Stone fragments</td><td>6</td><td>69,420–111,788</td></tr><tr><td>Clay fragments</td><td>19</td><td>2,380–69,908</td></tr><tr><td>Mortar fragments</td><td>11</td><td>4,168–102,498</td></tr><tr><td>Overall</td><td>36</td><td>2,380–111,788</td></tr></tbody></table>

Head), and mortar (Cake) objects. Objects were laser scanned and reconstructed via Poisson. Ground truth follows [1]. We selected 36 fragments using 75:25 train-test split stratified by material. Table I shows characteristics after Poisson reconstruction, exhibiting irregular topology typical of archaeological data. All meshes were remeshed to 3,000 faces (Algorithm 1).

## B. Evaluation Metrics

Segmentation quality was evaluated using region-based metrics adapted from the previous work [1]. For archaeological applications, region coherence matters more than exact boundary placement, as slightly misaligned boundaries may still yield usable surface regions for fragment alignment and reconstruction. We employ area overlap metrics that better capture segmentation utility for heritage preservation workflows than conventional edge-
level approaches.

### C. Setup

Implementation used Python 3.9 with PyTorch (CUDA 12.1) on an NVIDIA® GeForce RTX™ 3050Ti GPU (4GB memory) and Intel® Core™ i9-12900HK processor.

### D. Quantitative Results

Table II compares MS-DBSCAN (Section II) with MeshCNN using our preprocessing framework. MS-DBSCAN achieves 67.93% average region-based accuracy, validating that preprocessing preserves segmentation-relevant geometry. MeshCNN achieves 44.40%, demonstrating that edge-based deep learning can now operate on heritage data once meshes are standardized. The 23.5 percentage point gap reflects limited training data (27 fragments) rather than preprocessing failures—MS-DBSCAN requires no training while deep learning methods need substantial data to approach their capacity.

<table><caption>Bảng II: Region-based segmentation accuracy on archaeological test meshes.</caption><thead><tr><th rowspan="2">3D object</th><th colspan="2">Overall Accuracy</th></tr><tr><th>MS-DBSCAN</th><th>MeshCNN</th></tr></thead><tbody><tr><td>Brick part 05</td><td>0.8641</td><td>0.4318</td></tr><tr><td>Brick part 06</td><td>0.8025</td><td>0.4587</td></tr><tr><td>Cake part 09</td><td>0.5147</td><td>0.3178</td></tr><tr><td>Cake part 10</td><td>0.7059</td><td>0.3982</td></tr><tr><td>Cake part 11</td><td>0.7088</td><td>0.4232</td></tr><tr><td>Head part 10</td><td>0.7504</td><td>0.5801</td></tr><tr><td>Head part 11</td><td>0.5932</td><td>0.4915</td></tr><tr><td>Head part 12</td><td>0.5774</td><td>0.3793</td></tr><tr><td>Venus part 06</td><td>0.8195</td><td>0.5990</td></tr><tr><td>Venus part 07</td><td>0.4564</td><td>0.3605</td></tr><tr><td>Average</td><td>0.6793</td><td>0.4440</td></tr></tbody></table>

### E. Qualitative Results

Figure 2 shows example outputs from MS-DBSCAN and MeshCNN after preprocessing. MS-DBSCAN produces more stable region boundaries based on handcrafted geometric descriptors, while MeshCNN's learned features result in more fragmented predictions due to independent edge classification. Both methods successfully identify major surface regions, but differ in boundary precision and sensitivity to local geometric variations.

### F. Discussion

Preprocessing is essential for adapting irregular archaeological meshes to learning-based methods. Although MeshCNN does not yet surpass clustering, it shows that deep models can be trained once meshes are standardized. Unlike MS-DBSCAN, which depends on handcrafted features and parameters, MeshCNN learns representations directly from mesh connectivity and geometry.

Future work will focus on improving region coherence and applying few-shot or transfer learning to overcome dataset limitations. Together with preprocessing, these directions could establish deep learning as a viable complement to clustering for archaeological mesh segmentation.

## V. CONCLUSION

This paper established a preprocessing framework that makes MeshCNN-based deep learning feasible for archaeological surface segmentation. The framework addresses the incompatibility between irregular heritage meshes (2,380–111,788 faces, topological defects) and MeshCNN's requirements (fixed edge counts, manifold connectivity) through resolution normalization, topology validation, and label reprojection.

Experiments on 36 archaeological fragments validate the framework's effectiveness. MS-DBSCAN achieves 67.93% region-based accuracy on preprocessed meshes, demonstrating that the framework preserves segmentation-relevant geometry. MeshCNN achieves 44.40% accuracy with 27 training fragments, establishing that edge-based deep learning can now operate on heritage data. The 23.5 percentage point gap reflects limited training data rather than preprocessing failures—MS-DBSCAN requires no training while deep learning methods need substantial data.

The 3,000-face resolution balances MeshCNN's architectural requirements with geometric detail preservation and computational constraints. Since MeshCNN operates on mesh topology and edge features rather than material properties, this geometry-based normalization applies uniformly across stone, clay, and mortar fragments in our dataset.

Our contribution is not to demonstrate that MeshCNN outperforms clustering, but rather to establish the preprocessing methodology that makes such comparisons possible. Prior to this work, MeshCNN could not process archaeological meshes. The framework enables future research
<figure><img src="image_4.png" /><figcaption>Hình 2: Sample segmentation results on archaeological test objects (remeshed to 3,000 faces).</figcaption></figure>

to explore whether larger annotated datasets, few-shot learning techniques (prototypical networks, MAML), or transfer learning from synthetic fractured objects can close the performance gap. Future work will focus on these directions to reduce reliance on manual annotation while leveraging the preprocessing foundation established here.

## ACKNOWLEDGEMENT

This research was funded by the University of Science and Technology of Hanoi (USTH), Vietnam Academy of Science and Technology (VAST), under grant number "USTH.ICT.01/24-25."

## TÀI LIỆU

[1] V. N. Huynh, H. H. Nguyen, and R. Raffin, "A versatile multi-space dbscan framework for rough surface object segmentation," *Multimedia Tools and Applications*, pp. 1–25, 2025.

[2] R. Hanocka, A. Hertz, N. Fish, R. Giryes, S. Fleishman, and D. Cohen-Or, "MeshCNN: A network with an edge," *ACM Transactions on Graphics (TOG)*, vol. 38, no. 4, p. 90, 2019.

[3] Q.-X. Huang, S. Flöry, N. Gelfand, M. Hofer, and H. Pottmann, "Reassembling fractured objects by geometric matching," *ACM Trans. Graph.*, vol. 25, no. 3, p. 569–578, Jul. 2006. [Online]. Available: https://doi.org/10.1145/1141911.1141925

[4] A. Shamir, "A survey on mesh segmentation techniques," in *Computer graphics forum*, vol. 27, no. 6. Wiley Online Library, 2008, pp. 1539–1556.

[5] P. Theologou, I. Pratikakis, and T. Theoharis, "A comprehensive overview of methodologies and performance evaluation frameworks in 3d mesh segmentation," *Computer Vision and Image Understanding*, vol. 135, pp. 49–82, 2015.

[6] Y. He, H. Yu, X. Liu, Z. Yang, W. Sun, S. Anwar, and A. Mian, "Deep learning based 3d segmentation: A survey," *arXiv preprint arXiv:2103.05423*, 2021.

[7] J. Zhang, "Variational 3d mesh segmentation," Ph.D. dissertation, 2012.

[8] M. A. Uy, Q. Pham, B. Hua, D. T. Nguyen, and S. Yeung, "Revisiting point cloud classification: A new benchmark dataset and classification model on real-world data," *CoRR*, vol. abs/1908.04616, 2019. [Online]. Available: http://arxiv.org/abs/1908.04616

[9] S. Yang, M. Hou, and S. Li, "Three-dimensional point cloud semantic segmentation for cultural heritage: a comprehensive review," *Remote Sensing*, vol. 15, no. 3, p. 548, 2023.

[10] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, "Pointnet: Deep learning on point sets for 3d classification and segmentation," in *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 2017, pp. 652–660.

[11] D. Maturana and S. Scherer, "Voxnet: A 3d convolutional neural network for real-time object recognition," in *IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)*. IEEE, 2015, pp. 922–928.