# Strategies and Experiments for Massive 3D Digitalization of the Remains After the Notre-Dame de Paris' Fire

Florent Comte, Anthony Pamart, Kévin Réby, Livio De Luca

► To cite this version:

Florent Comte, Anthony Pamart, Kévin Réby, Livio De Luca. Strategies and Experiments for Massive 3D Digitalization of the Remains After the Notre-Dame de Paris' Fire. 10th International Workshop 3D-ARCH "3D Virtual Reconstruction and Visualization of Complex Architectures", Feb 2024, Sienna, Italy. pp.127-134, 10.5194/isprs-archives-XLVIII-2-W4-2024-127-2024. hal-04513698

HAL Id: hal-04513698

https://hal.science/hal-04513698

Submitted on 20 Mar 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Strategies and Experiments for Massive 3D Digitalization of the Remains After the Notre-Dame de Paris' Fire

Florent Comte, Anthony Pamart, Kévin Réby, Livio De Luca

► To cite this version:

Florent Comte, Anthony Pamart, Kévin Réby, Livio De Luca. Strategies and Experiments for Massive 3D Digitalization of the Remains After the Notre-Dame de Paris' Fire. 10th International Workshop 3D-ARCH "3D Virtual Reconstruction and Visualization of Complex Architectures", Feb 2024, Sienna, Italy. pp.127-134, 10.5194/isprs-archives-XLVIII-2-W4-2024-127-2024. hal-04513698

HAL Id: hal-04513698

https://hal.science/hal-04513698v1

Submitted on 20 Mar 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# STRATEGIES AND EXPERIMENTS FOR MASSIVE 3D DIGITALIZATION OF THE REMAINS AFTER THE NOTRE DAME DE PARIS' FIRE

Florent Comte¹, Anthony Pamart¹, Kévin Réby¹, Livio De Luca¹

¹UPR CNRS 2002 MAP, 13402 Marseille, France - (florent.comte, anthony.pamart, kevin.reby, livio.deluca)@map.cnrs.fr

## Commission II

**KEYWORDS:** Photogrammetry, Workflow automation, Deep learning, Segmentation, 3D Gaussian splitting, Cultural Heritage

## ABSTRACT:

After the catastrophic fire at Notre Dame de Paris, a significant challenge was presented by the numerous lead-contaminated remnants. To address this, a detailed digitization strategy was devised and executed, tailored to the unique needs of this extensive and diverse corpus. This strategy involved the development and customization of both hardware and software tools, ensuring their effectiveness throughout the digitization process – from initial data acquisition to data dissemination.

Central to our approach was the alignment of our methods with the distinct characteristics of each artifact, facilitating their effective preservation and future utility. Our strategy's adaptability was key, allowing us to incorporate advanced deep learning techniques into various aspects of our workflow. Notably, this included the implementation of the Segment Anything Model for automatic image segmentation, enhancing our image-based modeling capabilities. We also ventured into pioneering methods like 3D Gaussian Splatting and the exploration of radiance field methods for visualization.

Moreover, the project has been mindful of data responsibility, aiming to make all digital data openly accessible beyond 2025. We have placed a strong emphasis on harmonizing and managing data, minimizing redundancies, and ensuring efficient storage, all while maintaining transparency about the limitations and errors in our methodologies. This holistic approach to digitization, balancing technological innovation with responsible data management, aims to preserve and make accessible the digital heritage of Notre Dame de Paris for future generations.

# 1. INTRODUCTION

## 1.1. Context

Following the fire of April 15, 2019, which destroyed part of the cathedral Notre Dame de Paris, several working groups have been set up by the National Center for Scientific Research (CNRS) to help and organize the restoration work (Ball, 2020). This scientific action involves more than 175 researchers specialized in materials (stone, wood, metal, stained-glass), decoration, structural acoustics behaviors, and heritage emotions. Among them, the Digital Data Working Group (Groupe de Travail Données Numériques) is coordinating the development of a digital ecosystem aiming at managing the complete data lifecycle produced within the framework of the general scientific site (De Luca, 2024). The core technological framework of the digital ecosystem consists of a digital platform for the data centralisation (Néroulidis et al., 2024) including dedicated web services opened to more than 100 registered users for the ingestion, indexing and thematic categorisation of multimedia content, as well as for interactive analysis of 2D/3D/4D representations of architectural elements and remains. Since the beginning of the project, several temporal states of the cathedral (before-during-after the restoration) are digitized and integrated and semantically annotated with Aïoli (Abergel et al., 2023).

Within this framework, image-based digitization holds a sole role as a vector of interaction between on-site realities, methods, analyses, and interpretations, through 3D models but also (and most importantly), source images (high definition photographs) oriented in the correspondent 3D space. This ad-hoc framework allows us to enrich the 3D digitisation with a set of localized scientific observations that build a structure of overlapping and interconnected layers of thematic analyses.

Since 2019, Notre Dame's cathedral has been at the forefront of attention, but several thousand remains of the fire have been preserved (or recovered during archaeological and renovation operations), including timbers, vaults, metal artifacts, and

polychrome sculptures. This paper focuses on the comprehensive strategy we designed and implemented for the extensive image-based 3D digitization of these artifacts. The entire corpus consists of about ten thousand pieces of timber, over six hundred pallets of stone elements, and three hundred pallets of metal objects, documented on-site and stored in warehouses adapted to work safely in the context of lead pollution. (Zimmer et al., 2024).

## 1.2. Thinking on the overall strategy

The digitization endeavor serves several complementary goals. Firstly, the digital preservation of artifacts is crucial, especially since most items in this corpus are contaminated with lead and/or too delicate to handle without risk. Secondly, ensuring access to these digital collections for the diverse stakeholders involved (both in research and civil society) is a significant challenge in Digital Humanities, a field continuously evolving with technological advancements. Lastly, it's imperative that these collections serve as valuable research resources, meaning they should be comprehensively comparable in all their complexities and nuances.

The goals outlined are offset by certain "bottleneck" constraints intrinsic to this corpus, as highlighted in the previous section. Firstly, the sheer size of the corpus poses a challenge, encompassing several thousand artifacts that require digitization. Secondly, each artifact possesses unique characteristics, including material composition, surface treatment, functional scope, and conservation requirements. Finally, it's crucial that each digitization process is directly beneficial to researchers, with the Digital Data Working Group operating in close collaboration with other members of the Notre Dame scientific community.

Our operational approach is situated at the intersection of these objectives and the aforementioned constraints. The distinct nature of the Notre Dame corpus required the development of a comprehensive digitization strategy from an early stage. This strategy was designed to accommodate specific approaches for
# STRATEGIES AND EXPERIMENTS FOR MASSIVE 3D DIGITALIZATION OF THE REMAINS AFTER THE NOTRE DAME DE PARIS' FIRE

Florent Comte¹, Anthony Pamart¹, Kévin Réby¹, Livio De Luca¹

¹UPR CNRS 2002 MAP, 13402 Marseille, France - (florent.comte, anthony.pamart, kevin.reby, livio.deluca)@map.cnrs.fr

## Commission II

**KEYWORDS:** Photogrammetry, Workflow automation, Deep learning, Segmentation, 3D Gaussian splitting, Cultural Heritage

## ABSTRACT:

After the catastrophic fire at Notre Dame de Paris, a significant challenge was presented by the numerous lead-contaminated remnants. To address this, a detailed digitization strategy was devised and executed, tailored to the unique needs of this extensive and diverse corpus. This strategy involved the development and customization of both hardware and software tools, ensuring their effectiveness throughout the digitization process – from initial data acquisition to data dissemination.

Central to our approach was the alignment of our methods with the distinct characteristics of each artifact, facilitating their effective preservation and future utility. Our strategy's adaptability was key, allowing us to incorporate advanced deep learning techniques into various aspects of our workflow. Notably, this included the implementation of the Segment Anything Model for automatic image segmentation, enhancing our image-based modeling capabilities. We also ventured into pioneering methods like 3D Gaussian Splatting and the exploration of radiance field methods for visualization.

Moreover, the project has been mindful of data responsibility, aiming to make all digital data openly accessible beyond 2025. We have placed a strong emphasis on harmonizing and managing data, minimizing redundancies, and ensuring efficient storage, all while maintaining transparency about the limitations and errors in our methodologies. This holistic approach to digitization, balancing technological innovation with responsible data management, aims to preserve and make accessible the digital heritage of Notre Dame de Paris for future generations.

# 1. INTRODUCTION

## 1.1. Context

Following the fire of April 15, 2019, which destroyed part of the cathedral Notre Dame de Paris, several working groups have been set up by the National Center for Scientific Research (CNRS) to help and organize the restoration work (Ball, 2020). This scientific action involves more than 175 researchers specialized in materials (stone, wood, metal, stained-glass), decoration, structural acoustics behaviors, and heritage emotions. Among them, the Digital Data Working Group (Groupe de Travail Données Numériques) is coordinating the development of a digital ecosystem aiming at managing the complete data lifecycle produced within the framework of the general scientific site (De Luca, 2024). The core technological framework of the digital ecosystem consists of a digital platform for the data centralisation (Néroulidis et al., 2024) including dedicated web services opened to more than 100 registered users for the ingestion, indexing and thematic categorisation of multimedia content, as well as for interactive analysis of 2D/3D/4D representations of architectural elements and remains. Since the beginning of the project, several temporal states of the cathedral (before-during-after the restoration) are digitized and integrated and semantically annotated with Aïoli (Abergel et al., 2023).

Within this framework, image-based digitization holds a sole role as a vector of interaction between on-site realities, methods, analyses, and interpretations, through 3D models but also (and most importantly), source images (high definition photographs) oriented in the correspondent 3D space. This ad-hoc framework allows us to enrich the 3D digitisation with a set of localized scientific observations that build a structure of overlapping and interconnected layers of thematic analyses.

Since 2019, Notre Dame's cathedral has been at the forefront of attention, but several thousand remains of the fire have been preserved (or recovered during archaeological and renovation operations), including timbers, vaults, metal artifacts, and

polychrome sculptures. This paper focuses on the comprehensive strategy we designed and implemented for the extensive image-based 3D digitization of these artifacts. The entire corpus consists of about ten thousand pieces of timber, over six hundred pallets of stone elements, and three hundred pallets of metal objects, documented on-site and stored in warehouses adapted to work safely in the context of lead pollution. (Zimmer et al., 2024).

## 1.2. Thinking on the overall strategy

The digitization endeavor serves several complementary goals. Firstly, the digital preservation of artifacts is crucial, especially since most items in this corpus are contaminated with lead and/or too delicate to handle without risk. Secondly, ensuring access to these digital collections for the diverse stakeholders involved (both in research and civil society) is a significant challenge in Digital Humanities, a field continuously evolving with technological advancements. Lastly, it's imperative that these collections serve as valuable research resources, meaning they should be comprehensively comparable in all their complexities and nuances.

The goals outlined are offset by certain "bottleneck" constraints intrinsic to this corpus, as highlighted in the previous section. Firstly, the sheer size of the corpus poses a challenge, encompassing several thousand artifacts that require digitization. Secondly, each artifact possesses unique characteristics, including material composition, surface treatment, functional scope, and conservation requirements. Finally, it's crucial that each digitization process is directly beneficial to researchers, with the Digital Data Working Group operating in close collaboration with other members of the Notre Dame scientific community.

Our operational approach is situated at the intersection of these objectives and the aforementioned constraints. The distinct nature of the Notre Dame corpus required the development of a comprehensive digitization strategy from an early stage. This strategy was designed to accommodate specific approaches for
each group of items, also taking into account unique scientific
and heritage requirements. Our overarching strategy is anchored
in the following key concepts (as illustrated in Figure 1), which
allow us to navigate through constraints while steadfastly
adhering to our primary objectives:

* Optimizing Data Acquisition and Processing: This aspect was meticulously planned to minimize delays in scientific studies. Our approach involves working synergistically with other teams operating concurrently, ensuring that our activities complement rather than hinder the broader research efforts.

* Accommodating a Wide Dissemination: The diversity in potential applications, ranging from model analysis to public visualization, is aligned with the evolving realities of practice and technical advancements. This broad scope necessitates a flexible approach to cater to various requirements.

* Harmonizing and Managing Heterogeneous Data: The strategy for dealing with the inherently diverse nature of the data revolves around ontological principles. This includes the use of enhanced metadata and paradata, ensuring consistency and accessibility across the dataset.

<figure><img src="image_2.png" /><figcaption>Figure 1. Overall strategy diagram</figcaption></figure>

Our overarching strategy was adapted to align with the specific expertise of our team. For instance, in advocating for consistent data management, we opted for a singular digitization approach. Image-based modeling was selected, not only for its long-established use in our team's research (as detailed in works by De Luca et al., 2022; De Luca, 2023; Pamart et al., 2020), but also due to its recognized benefits, which we will discuss further below. Moreover, as previously noted, our strategy required customization to accommodate the diverse nature of the corpus. This adaptation was based on several intrinsic characteristics of the artifacts, including:

* Morphology: Considering material, size, weight, and shape;

* Conservation Conditions: Assessing factors such as pollution or surface treatments like traces of polychromy.

* Access Conditions: Evaluating aspects like responsibility, storage location, and pollution levels of the area.

* Intended Scope of Use: Determining if our acquisition solutions are optimally calibrated, ensuring they are neither over- nor under-utilized.

A research endeavor like ours is subject to various evolutions, a fact we anticipated from the beginning. This foresight allowed for flexibility, fostering innovation in several areas, particularly in the integration of deep learning solutions into our digitization strategy. This approach facilitates the exploration of diverse

alternatives, enhancing both the optimization and dissemination
of our protocols and data.

The exploration of deep learning serves a dual purpose:

* Integration with Current Data Production: We aim to discover how these advanced tools can effectively complement and enhance our existing data production processes.

* Testing with Real Heritage Data: Unlike relying solely on academic benchmarks or synthetic data, we are testing these tools with actual heritage data. This approach provides a more authentic and practical understanding of their capabilities and limitations in real-world scenarios.

## 2. OPTIMIZATION STRATEGIES

The term "optimization" in our context refers to identifying solutions and methods that enhance the efficiency, preservation, and management of our corpus. This also includes facilitating communication with other members of Notre Dame's scientific group, while aligning with the concurrent timing of scientific studies and digitization efforts. We have categorized this optimization endeavor into several key segments. These are primarily adaptations of standard photogrammetry digitization protocols:

Thus, we can divide this optimization effort into several segments, which are, for the most part, derived from standard image-based digitization protocols: (1) optimization of data acquisition and calibration; (2) optimization of dense matching processing and export of intended models (via Agisoft Metashape's python API), optimization of long-term archiving and data management; (3) experimentation with optimized processing protocol through automatic mask generation for each image collection using deep learning (via Segmentation Anything Model).

### 2.1. Acquisition and calibration challenges

From the outset, the task of acquiring photogrammetric data for the Notre-Dame corpus presented significant complexities. The lead pollution coating the objects has been a notable issue. Moreover, the majority of the artifacts can be characterized as both fragile and cumbersome. An additional challenge lies in the time constraints, necessitating a high level of efficiency in terms of manpower per data acquisition campaign. For instance, we undertook a rapid and urgent data acquisition campaign for the 40 remaining pieces of the oculus. Each piece required handling by multiple operators, emphasizing the need for speed and efficiency. This was made feasible within a narrow time frame through the deployment of a lightweight photogrammetric studio, as illustrated in Figure 2 (top-left). This approach exemplifies the need for agility and precision in our acquisition and calibration processes, particularly under challenging conditions.

To optimize the acquisition process, we established a partnership with Mercurio Imaging, a specialist company based in Marseille. This collaboration led to the creation of three innovative devices, each tailored to specific needs of our digitization project:

Dome-Shaped Rig (Figure 2 - top-right): This rig was developed to facilitate data acquisition on the complex remains of the transept crossing's double-arched keystones, as detailed in Guillem et al., 2023. Its dome shape is particularly suited for capturing detailed images from multiple angles.

Multi-Camera Studio (Figure 2 - bottom-left): Designed as an alternative to the dome, this studio is particularly effective for digitizing smaller artifacts, such as metal staples. Its
each group of items, also taking into account unique scientific
and heritage requirements. Our overarching strategy is anchored
in the following key concepts (as illustrated in Figure 1), which
allow us to navigate through constraints while steadfastly
adhering to our primary objectives:

* Optimizing Data Acquisition and Processing: This aspect was meticulously planned to minimize delays in scientific studies. Our approach involves working synergistically with other teams operating concurrently, ensuring that our activities complement rather than hinder the broader research efforts.

* Accommodating a Wide Dissemination: The diversity in potential applications, ranging from model analysis to public visualization, is aligned with the evolving realities of practice and technical advancements. This broad scope necessitates a flexible approach to cater to various requirements.

* Harmonizing and Managing Heterogeneous Data: The strategy for dealing with the inherently diverse nature of the data revolves around ontological principles. This includes the use of enhanced metadata and paradata, ensuring consistency and accessibility across the dataset.

<figure><img src="image_2.png" /><figcaption>Figure 1. Overall strategy diagram</figcaption></figure>

Our overarching strategy was adapted to align with the specific expertise of our team. For instance, in advocating for consistent data management, we opted for a singular digitization approach. Image-based modeling was selected, not only for its long-established use in our team's research (as detailed in works by De Luca et al., 2022; De Luca, 2023; Pamart et al., 2020), but also due to its recognized benefits, which we will discuss further below. Moreover, as previously noted, our strategy required customization to accommodate the diverse nature of the corpus. This adaptation was based on several intrinsic characteristics of the artifacts, including:

* Morphology: Considering material, size, weight, and shape;

* Conservation Conditions: Assessing factors such as pollution or surface treatments like traces of polychromy.

* Access Conditions: Evaluating aspects like responsibility, storage location, and pollution levels of the area.

* Intended Scope of Use: Determining if our acquisition solutions are optimally calibrated, ensuring they are neither over- nor under-utilized.

A research endeavor like ours is subject to various evolutions, a fact we anticipated from the beginning. This foresight allowed for flexibility, fostering innovation in several areas, particularly in the integration of deep learning solutions into our digitization strategy. This approach facilitates the exploration of diverse

alternatives, enhancing both the optimization and dissemination
of our protocols and data.

The exploration of deep learning serves a dual purpose:

*   **Integration with Current Data Production:** We aim to discover how these advanced tools can effectively complement and enhance our existing data production processes.

* Testing with Real Heritage Data: Unlike relying solely on academic benchmarks or synthetic data, we are testing these tools with actual heritage data. This approach provides a more authentic and practical understanding of their capabilities and limitations in real-world scenarios.

## 2. OPTIMIZATION STRATEGIES

The term "optimization" in our context refers to identifying solutions and methods that enhance the efficiency, preservation, and management of our corpus. This also includes facilitating communication with other members of Notre Dame's scientific group, while aligning with the concurrent timing of scientific studies and digitization efforts. We have categorized this optimization endeavor into several key segments. These are primarily adaptations of standard photogrammetry digitization protocols:

Thus, we can divide this optimization effort into several segments, which are, for the most part, derived from standard image-based digitization protocols: (1) optimization of data acquisition and calibration; (2) optimization of dense matching processing and export of intended models (via Agisoft Metashape's python API), optimization of long-term archiving and data management; (3) experimentation with optimized processing protocol through automatic mask generation for each image collection using deep learning (via Segmentation Anything Model).

### 2.1. Acquisition and calibration challenges

From the outset, the task of acquiring photogrammetric data for the Notre-Dame corpus presented significant complexities. The lead pollution coating the objects has been a notable issue. Moreover, the majority of the artifacts can be characterized as both fragile and cumbersome. An additional challenge lies in the time constraints, necessitating a high level of efficiency in terms of manpower per data acquisition campaign. For instance, we undertook a rapid and urgent data acquisition campaign for the 40 remaining pieces of the oculus. Each piece required handling by multiple operators, emphasizing the need for speed and efficiency. This was made feasible within a narrow time frame through the deployment of a lightweight photogrammetric studio, as illustrated in Figure 2 (top-left). This approach exemplifies the need for agility and precision in our acquisition and calibration processes, particularly under challenging conditions.

To optimize the acquisition process, we established a partnership with Mercurio Imaging, a specialist company based in Marseille. This collaboration led to the creation of three innovative devices, each tailored to specific needs of our digitization project:

Dome-Shaped Rig (Figure 2 - top-right): This rig was developed to facilitate data acquisition on the complex remains of the transect crossing's double-arched keystones, as detailed in Guillem et al., 2023. Its dome shape is particularly suited for capturing detailed images from multiple angles.

Multi-Camera Studio (Figure 2 - bottom-left): Designed as an alternative to the dome, this studio is particularly effective for digitizing smaller artifacts, such as metal staples. Its
multi-camera setup allows for comprehensive coverage,
ensuring detailed and accurate digitization.

Step-Over, an Arch-Shape In-Line Photogrammetric Structure (Figure 2 - bottom-right): This structure is specially designed for efficiently digitizing artifacts that are elongated along a specific axis, like timbers. Its arch shape and in-line configuration allow for quick and thorough data capture.

<figure><img src="image_3.png" /><figcaption>**Figure 2.** Optimized set-ups (top-left: flash synchronized photogrammetry; top-right: "dome" automatic machine, made for heavy remains; bottom-left: "studio" automatic machine, made for tiny artifacts; bottom-right: "step-over" semi-automatic machine, made for timber)</figcaption></figure>

Our photogrammetric protocol is practically implemented
through a method we term "half-shell" acquisition. This
approach involves rotating the object to capture all its surfaces

through multiple complementary acquisitions, termed as "half-shells." These half-shells are then meticulously merged to form a "complete shell" at the conclusion of the photogrammetric processing. For each half-shell acquisition, we employ coded markers in the form of horizontal squares. These markers serve a dual purpose: they establish the scale and define the horizontal plane of the object. This setup is crucial for automating the subsequent processing stages, ensuring accuracy and efficiency in the digitization process. The integration of these markers into our workflow exemplifies the attention to detail and precision required in high-quality heritage digitization.

## 2.2. Photogrammetric processing enhancement

Once we had successfully acquired and organized the source
image collections, it became evident that software processing
required similar optimization. We opted for Agisoft Metashape,
a proprietary software, which is popular in Cultural Heritage
digital documentation. The software was adapted early for
intensive GPU use, enhancing its speed, and it features an
interface integrated with a Python API that covers all its
processes. This integration allowed us to create specific
commands for the automatic processing of all acquisitions in
our corpus, addressing the unique aspects of the image
collections. We focused on color calibration, as outlined in
Gaiani and Bellabeni (2018), and format conversion. This
involved establishing a connection between our Python
commands and external tools such as Shaft2-CLI and
Rawtherapee-CLI. In addition to these steps, our process
included the use of coded markers for retrieving the scale and
establishing the horizontal plane, modeling each half-shell, and
ultimately merging all the half-shells to produce the final
output, as shown in Figure 3.

<figure><img src="image_4.png" /><figcaption>**Figure 3.** Automatic image-based digitization workflow used for Notre-Dame corpus, from acquisition specificities to deep learning improvements.</figcaption></figure>
multi-camera setup allows for comprehensive coverage,
ensuring detailed and accurate digitization.

Step-Over, an Arch-Shape In-Line Photogrammetric Structure (Figure 2 - bottom-right): This structure is specially designed for efficiently digitizing artifacts that are elongated along a specific axis, like timbers. Its arch shape and in-line configuration allow for quick and thorough data capture.

<figure><img src="image_3.png" /><figcaption>**Figure 2.** Optimized set-ups (top-left: flash synchronized photogrammetry; top-right: "dome" automatic machine, made for heavy remains; bottom-left: "studio" automatic machine, made for tiny artifacts; bottom-right: "step-over" semi-automatic machine, made for timber)</figcaption></figure>

Our photogrammetric protocol is practically implemented
through a method we term "half-shell" acquisition. This
approach involves rotating the object to capture all its surfaces

through multiple complementary acquisitions, termed as "half-shells." These half-shells are then meticulously merged to form a "complete shell" at the conclusion of the photogrammetric processing. For each half-shell acquisition, we employ coded markers in the form of horizontal squares. These markers serve a dual purpose: they establish the scale and define the horizontal plane of the object. This setup is crucial for automating the subsequent processing stages, ensuring accuracy and efficiency in the digitization process. The integration of these markers into our workflow exemplifies the attention to detail and precision required in high-quality heritage digitization.

## 2.2. Photogrammetric processing enhancement

Once we had successfully acquired and organized the source
image collections, it became evident that software processing
required similar optimization. We opted for Agisoft Metashape,
a proprietary software, which is popular in Cultural Heritage
digital documentation. The software was adapted early for
intensive GPU use, enhancing its speed, and it features an
interface integrated with a Python API that covers all its
processes. This integration allowed us to create specific
commands for the automatic processing of all acquisitions in
our corpus, addressing the unique aspects of the image
collections. We focused on color calibration, as outlined in
Gaiani and Bellabeni (2018), and format conversion. This
involved establishing a connection between our Python
commands and external tools such as Shaft2-CLI and
Rawtherapee-CLI. In addition to these steps, our process
included the use of coded markers for retrieving the scale and
establishing the horizontal plane, modeling each half-shell, and
ultimately merging all the half-shells to produce the final
output, as shown in Figure 3.

<figure><img src="image_4.png" /><figcaption>**Figure 3.** Automatic image-based digitization workflow used for Notre-Dame corpus, from acquisition specificities to deep learning improvements.</figcaption></figure>
In practice, processing each half-shell is a meticulous task. It not only involves accurately recording the intrinsic parameters and camera orientations but also entails generating masks from the horizontal plane, which are marked with coded markers. These masks play a crucial role in the process, as they allow for the isolation of the object from its support. This isolation is key for correctly aligning the half-shells with each other during the merging process. It helps to identify the relevant correspondences while avoiding false correspondences that might arise from the support, which could be redundantly present in each image. After completing the final mesh, we can export it from Metashape in various formats. This versatility in exporting is crucial for the next phase, which we will discuss in more detail later. The export formats range from standard ones like .obj and .ply to more specific or software-oriented formats such as potree, 3DHOP, and Aioli. In cases of these specific formats, we integrate our Python Metashape commands with external converters provided by their creators (NexusConverter.exe, PotreeConverter.exe, MS2AIOLI.py<fnref n="1" />).

## 2.3. SAM

The Segment Anything Model (SAM) is a cutting-edge image segmentation foundation model developed by Meta, as outlined by Kirillov et al. in 2023. As a foundation model, SAM is a large deep neural network trained on an extensive dataset, typically through unsupervised learning. This training enables it to be adapted for a wide array of tasks, as discussed by Bommasani et al. in 2021. SAM's training involved the massive SA-1B dataset, which comprises over 1 billion instance masks across 11 million images. This vast and varied dataset endows SAM with exceptional generalization capabilities, allowing it to perform zero-shot segmentation. This means SAM can segment objects and regions in images without needing prior training on specific objects or categories. SAM operates by producing a segmentation mask in response to a prompt, which could be a set of points, bounding boxes, or text. In our photogrammetry pipeline, we utilize images with bounding boxes as inputs for SAM to generate binary segmentation masks, which are then integrated into our pipeline. SAM's architecture consists of three core modules:

*   **Image Encoder:** Utilizing a MaskedAutoEncoder (MAE) based on a Vision Transformer (ViT) model, as proposed by He et al. in 2022 and Dosovitskiy et al. in 2020, this module is responsible for image feature extraction. It processes an image to create an image embedding.

*   **Prompt Encoder:** This module can process positional information from the prompt input, such as a set of points and/or bounding boxes. These inputs provide additional contextual information for the mask decoding process.

*   **Mask Decoder:** A two-layer transformer-based model, this module is tasked with generating the final segmentation mask predictions.

This key finding from our previous research highlights the effectiveness of SAM, particularly when using bounding boxes as prompts. This method leverages SAM's advanced capabilities, demonstrating its adaptability and precision in complex image segmentation tasks. The use of bounding boxes provides clear, defined areas for SAM to focus on, enhancing the accuracy of the segmentation process (Figure 4).

<figure><img src="image_5.png" /><figcaption>**Figure 4.** Overview of SAM for automatic segmentation.</figcaption></figure>

## 3. HARMONIZATION PROCESS

In our digitization project, each artifact presents unique characteristics, such as shape, material, and specific treatments, often detailed in scientific studies. Similarly, their virtual counterparts also possess distinctive features related to their creation or topological aspects. Beyond these, the digital models hold contextual specifics: the location and date of image acquisition, the software used, and the operator involved. Digitization, as a scientific endeavor, generates a collection of resources (data) following a technical protocol (metadata), while also preserving a spatio-temporal uniqueness (paradata). The final model, a product of this combination, should accurately represent these elements and be capable of being queried in its full complexity. To achieve this, a harmonization of the data/metadata/paradata package is essential. This process ensures that the data is not only comparable within our project but also interoperable with external systems and projects. Harmonization plays a critical role in maintaining consistency, reliability, and usability of the digitized data, making it a pivotal aspect of our work in digital preservation and research.

### 3.1. Data, Metadata and Paradata integration

In our effort to harmonize and ensure the interoperability of our data, we have integrated a paradata container into each stage of our workflow, drawing on the MEMoS system (Pamart et al., 2022). MEMoS, based on the W7 system (Liu and Ram, 2017), facilitates the embedding of paradata within the images via a QR code. This code encapsulates contextual information across seven categories: What, When, Where, Why, Who, Which, and hoW. We have adopted this classification to systematically document the digitization activity's paradata in several complementary files, each dedicated to different contexts: acquisition, processing, and data management. Moreover, we diligently maintain the metadata for both the source and calibrated images, as well as preserving the Agisoft Metashape projects. This approach includes defining the boundaries of our work by exporting various indicators, particularly those related to spatial orientation uncertainties. These indicators, such as RMS error per camera, SEUW error, and homothety uncertainty, are crucial. They not only allow us to assess the performance level of our operational workflow for each digitization campaign but also aid in identifying any projects that encountered issues.

### 3.2. Data management

Our workflow is designed to manage a vast array of complementary data efficiently, aiming to organize them systematically from the outset to prevent the creation of unmanageable digital bulk. Our approach to data management is

<footnote n="1">¹ https://gitlab.huma-num.fr/apamart/ms2aioli</footnote>
In practice, processing each half-shell is a meticulous task. It not only involves accurately recording the intrinsic parameters and camera orientations but also entails generating masks from the horizontal plane, which are marked with coded markers. These masks play a crucial role in the process, as they allow for the isolation of the object from its support. This isolation is key for correctly aligning the half-shells with each other during the merging process. It helps to identify the relevant correspondences while avoiding false correspondences that might arise from the support, which could be redundantly present in each image. After completing the final mesh, we can export it from Metashape in various formats. This versatility in exporting is crucial for the next phase, which we will discuss in more detail later. The export formats range from standard ones like .obj and .ply to more specific or software-oriented formats such as potree, 3DHOP, and Aioli. In cases of these specific formats, we integrate our Python Metashape commands with external converters provided by their creators (NexusConverter.exe, PotreeConverter.exe, MS2AIOLI.py<fnref n="1" />).

## 2.3. SAM

The Segment Anything Model (SAM) is a cutting-edge image segmentation foundation model developed by Meta, as outlined by Kirillov et al. in 2023. As a foundation model, SAM is a large deep neural network trained on an extensive dataset, typically through unsupervised learning. This training enables it to be adapted for a wide array of tasks, as discussed by Bommasani et al. in 2021. SAM's training involved the massive SA-1B dataset, which comprises over 1 billion instance masks across 11 million images. This vast and varied dataset endows SAM with exceptional generalization capabilities, allowing it to perform zero-shot segmentation. This means SAM can segment objects and regions in images without needing prior training on specific objects or categories. SAM operates by producing a segmentation mask in response to a prompt, which could be a set of points, bounding boxes, or text. In our photogrammetry pipeline, we utilize images with bounding boxes as inputs for SAM to generate binary segmentation masks, which are then integrated into our pipeline. SAM's architecture consists of three core modules:

*   **Image Encoder:** Utilizing a MaskedAutoEncoder (MAE) based on a Vision Transformer (ViT) model, as proposed by He et al. in 2022 and Dosovitskiy et al. in 2020, this module is responsible for image feature extraction. It processes an image to create an image embedding.

*   **Prompt Encoder:** This module can process positional information from the prompt input, such as a set of points and/or bounding boxes. These inputs provide additional contextual information for the mask decoding process.

*   **Mask Decoder:** A two-layer transformer-based model, this module is tasked with generating the final segmentation mask predictions.

This key finding from our previous research highlights the effectiveness of SAM, particularly when using bounding boxes as prompts. This method leverages SAM's advanced capabilities, demonstrating its adaptability and precision in complex image segmentation tasks. The use of bounding boxes provides clear, defined areas for SAM to focus on, enhancing the accuracy of the segmentation process (Figure 4).

<figure><img src="image_5.png" /><figcaption>**Figure 4.** Overview of SAM for automatic segmentation.</figcaption></figure>

## 3. HARMONIZATION PROCESS

In our digitization project, each artifact presents unique characteristics, such as shape, material, and specific treatments, often detailed in scientific studies. Similarly, their virtual counterparts also possess distinctive features related to their creation or topological aspects. Beyond these, the digital models hold contextual specifics: the location and date of image acquisition, the software used, and the operator involved. Digitization, as a scientific endeavor, generates a collection of resources (data) following a technical protocol (metadata), while also preserving a spatio-temporal uniqueness (paradata). The final model, a product of this combination, should accurately represent these elements and be capable of being queried in its full complexity. To achieve this, a harmonization of the data/metadata/paradata package is essential. This process ensures that the data is not only comparable within our project but also interoperable with external systems and projects. Harmonization plays a critical role in maintaining consistency, reliability, and usability of the digitized data, making it a pivotal aspect of our work in digital preservation and research.

### 3.1. Data, Metadata and Paradata integration

In our effort to harmonize and ensure the interoperability of our data, we have integrated a paradata container into each stage of our workflow, drawing on the MEMoS system (Pamart et al., 2022). MEMoS, based on the W7 system (Liu and Ram, 2017), facilitates the embedding of paradata within the images via a QR code. This code encapsulates contextual information across seven categories: What, When, Where, Why, Who, Which, and hoW. We have adopted this classification to systematically document the digitization activity's paradata in several complementary files, each dedicated to different contexts: acquisition, processing, and data management. Moreover, we diligently maintain the metadata for both the source and calibrated images, as well as preserving the Agisoft Metashape projects. This approach includes defining the boundaries of our work by exporting various indicators, particularly those related to spatial orientation uncertainties. These indicators, such as RMS error per camera, SEUW error, and homothety uncertainty, are crucial. They not only allow us to assess the performance level of our operational workflow for each digitization campaign but also aid in identifying any projects that encountered issues.

### 3.2. Data management

Our workflow is designed to manage a vast array of complementary data efficiently, aiming to organize them systematically from the outset to prevent the creation of unmanageable digital bulk. Our approach to data management is

<footnote n="1">¹ https://gitlab.huma-num.fr/apamart/ms2aioli</footnote>
straightforward yet effective, with each digitization project
encapsulated in a structured folder hierarchy:

*   **Acquisition:** This contains the source images. Within it, a MEMoS file in JSON format documents the paradata from the acquisition phase.

* Processing: This includes the Metashape folder, converted and calibrated images (in JPG or TIF formats), and a processing MEMoS file describing the workflow. This folder also stores data on digitization uncertainties.

* Export: This folder houses each of the virtual model formats generated during the digitization process.

*   **Long-Term Archive:** It contains the Submission Information Package (SIP) for long-term archiving purposes.

At the pinnacle of this hierarchy is a general index that provides an overview of each folder. Crucially, this index establishes a link between the virtual model and the physical artifact, referencing the perpetual inventory number assigned by the scientific site, as illustrated in Figure 5.

<figure><img src="image_6.png" /><figcaption>**Figure 5.** Data structure for each item</figcaption></figure>

This artifact-specific data structure is generated automatically, at the same time as each MEMoS, based on information pre-filled by the various operators. In this way, we try to minimize the risk of error in our data management.

## 4. DISSEMINATION STRATEGIES

The dissemination of Notre Dame's digitized artifacts is envisioned to encompass a broad spectrum of applications. These range from long-term digital preservation to the practical utilization of virtual models, and also include making these resources accessible to the public. Public engagement could be achieved through online open-source visualization platforms or through immersive experiences like Virtual Reality solutions. Furthermore, an essential aspect of this dissemination process is to ensure that these digitizations contribute to the AIOLI platform. This platform plays a pivotal role in analyzing and preserving the complex scientific discourse of Notre Dame's scientific group.

*   **Complexity and Scientific Value:** The digitization process addresses the need to capture the complexity of the artifacts and the associated scientific discourse. This is particularly crucial in the heritage discipline, where data is inherently fragmentary and heterogeneous.

*   **Responsibility and Reliability:** In the field of scientific heritage, the responsibility of data providers extends to ensuring traceability and reliability. This is vital due to the fragmentary and diverse nature of heritage data.

*   **Technological Innovation and Experimentation:** The dissemination process is also an opportunity for technological experimentation and advancement. One such area is the use of Radiance Field methods. These methods

can be useful in documentation, visualization, and communication, offering new ways to present 3D models of cultural heritage sites and objects.

This multi-faceted approach to dissemination not only preserves the heritage value of the artifacts but also opens up new avenues for research, education, and public engagement.

### 4.1. Visualizations, measurements, annotations and preservation techniques

For our project, we have carefully selected file formats for exporting digitized artifacts to ensure compatibility with a wide range of processing, analysis, and visualization tools.

*   **Dense Point Cloud export:** We export the dense point cloud in the Stanford Triangle Format (.PLY). This format is recognized by both processing software like CloudCompare and Meshlab, and visualization software through plugins for Blender and Unity. The .PLY format is versatile, allowing for the storage and conversion of various properties within the point cloud into other formats, such as .LAZ and E57. Its widespread use in the Digital Heritage community further validates our choice.

*   **Mesh export:** For meshes, we have chosen the Wavefront OBJ format, which is commonly used in processing and analysis (CloudCompare, Meshlab) as well as in modeling software (Blender, Unity). This format has evolved to include Physically Based Rendering (PBR) specifications within the .MTL parameters, making it compatible with modern rendering engines.

*   **Online Visualization:** We generate two types of online visualizations, each as a self-sufficient webpage containing all necessary libraries. The dense cloud is accessible in the Potree format (version 2.1, selected to avoid the file multiplicity of version 1.7), while the mesh is provided in the nexus compressed format (.NXZ) for the 3DHOP viewer. This approach ensures that each artifact's visualization is autonomous, allowing for the creation of specific collections either by linking to our generated .html pages or, for more integrated solutions, directly to the potree/nexus model itself.

Our export workflow extends beyond simple 3D asset generation; it includes creating a ready-to-use AIOLI project, particularly noteworthy for its functionalities :

*   **Visualization of the Dense Cloud:** It provides an interactive display of the dense cloud, facilitating detailed examination and analysis.

*   **Spatially Oriented Image Visualization:** More importantly, it allows for the visualization of each spatially oriented image.

*   **Multi-layered 2D/3D annotation:** This feature is crucial for complex and precise annotations of the virtual models directly through the images. These annotations are reprojected onto the complete collection, utilizing photogrammetry's intrinsic parameters and camera positions (computed by an internal integration of the MicMac photogrammetric runtime).

In line with the French national digital preservation policy, our project is evolving within the TGIR Huma-Num and the Archeogrid platform (Tournon et al., 2020) for data storage, referencing, and long-term archiving. This collaboration includes creating a bridge between our solutions and the aLTAG3D software. This enables us to generate a Submission Information Package (SIP) for each item, comprising a collection of archives formatted according to a specific archival profile. These packages are then deposited on the national
straightforward yet effective, with each digitization project
encapsulated in a structured folder hierarchy:

*   **Acquisition:** This contains the source images. Within it, a MEMoS file in JSON format documents the paradata from the acquisition phase.

* Processing: This includes the Metashape folder, converted and calibrated images (in JPG or TIF formats), and a processing MEMoS file describing the workflow. This folder also stores data on digitization uncertainties.

* Export: This folder houses each of the virtual model formats generated during the digitization process.

*   **Long-Term Archive:** It contains the Submission Information Package (SIP) for long-term archiving purposes.

At the pinnacle of this hierarchy is a general index that provides an overview of each folder. Crucially, this index establishes a link between the virtual model and the physical artifact, referencing the perpetual inventory number assigned by the scientific site, as illustrated in Figure 5.

<figure><img src="image_6.png" /><figcaption>**Figure 5.** Data structure for each item</figcaption></figure>

This artifact-specific data structure is generated automatically, at the same time as each MEMoS, based on information pre-filled by the various operators. In this way, we try to minimize the risk of error in our data management.

# 4. DISSEMINATION STRATEGIES

The dissemination of Notre Dame's digitized artifacts is envisioned to encompass a broad spectrum of applications. These range from long-term digital preservation to the practical utilization of virtual models, and also include making these resources accessible to the public. Public engagement could be achieved through online open-source visualization platforms or through immersive experiences like Virtual Reality solutions. Furthermore, an essential aspect of this dissemination process is to ensure that these digitizations contribute to the AIOLI platform. This platform plays a pivotal role in analyzing and preserving the complex scientific discourse of Notre Dame's scientific group.

*   **Complexity and Scientific Value:** The digitization process addresses the need to capture the complexity of the artifacts and the associated scientific discourse. This is particularly crucial in the heritage discipline, where data is inherently fragmentary and heterogeneous.

*   **Responsibility and Reliability:** In the field of scientific heritage, the responsibility of data providers extends to ensuring traceability and reliability. This is vital due to the fragmentary and diverse nature of heritage data.

*   **Technological Innovation and Experimentation:** The dissemination process is also an opportunity for technological experimentation and advancement. One such area is the use of Radiance Field methods. These methods

can be useful in documentation, visualization, and communication, offering new ways to present 3D models of cultural heritage sites and objects.

This multi-faceted approach to dissemination not only preserves the heritage value of the artifacts but also opens up new avenues for research, education, and public engagement.

## 4.1. Visualizations, measurements, annotations and preservation techniques

For our project, we have carefully selected file formats for exporting digitized artifacts to ensure compatibility with a wide range of processing, analysis, and visualization tools.

*   **Dense Point Cloud export:** We export the dense point cloud in the Stanford Triangle Format (.PLY). This format is recognized by both processing software like CloudCompare and Meshlab, and visualization software through plugins for Blender and Unity. The .PLY format is versatile, allowing for the storage and conversion of various properties within the point cloud into other formats, such as .LAZ and E57. Its widespread use in the Digital Heritage community further validates our choice.

*   **Mesh export:** For meshes, we have chosen the Wavefront OBJ format, which is commonly used in processing and analysis (CloudCompare, Meshlab) as well as in modeling software (Blender, Unity). This format has evolved to include Physically Based Rendering (PBR) specifications within the .MTL parameters, making it compatible with modern rendering engines.

*   **Online Visualization:** We generate two types of online visualizations, each as a self-sufficient webpage containing all necessary libraries. The dense cloud is accessible in the Potree format (version 2.1, selected to avoid the file multiplicity of version 1.7), while the mesh is provided in the nexus compressed format (.NXZ) for the 3DHOP viewer. This approach ensures that each artifact's visualization is autonomous, allowing for the creation of specific collections either by linking to our generated .html pages or, for more integrated solutions, directly to the potree/nexus model itself.

Our export workflow extends beyond simple 3D asset generation; it includes creating a ready-to-use AIOLI project, particularly noteworthy for its functionalities :

*   **Visualization of the Dense Cloud:** It provides an interactive display of the dense cloud, facilitating detailed examination and analysis.

*   **Spatially Oriented Image Visualization:** More importantly, it allows for the visualization of each spatially oriented image.

*   **Multi-layered 2D/3D annotation:** This feature is crucial for complex and precise annotations of the virtual models directly through the images. These annotations are reprojected onto the complete collection, utilizing photogrammetry's intrinsic parameters and camera positions (computed by an internal integration of the MicMac photogrammetric runtime).

In line with the French national digital preservation policy, our project is evolving within the TGIR Huma-Num and the Archeogrid platform (Tournon et al., 2020) for data storage, referencing, and long-term archiving. This collaboration includes creating a bridge between our solutions and the aLTAG3D software. This enables us to generate a Submission Information Package (SIP) for each item, comprising a collection of archives formatted according to a specific archival profile. These packages are then deposited on the national
digital archiving service provided by the CINES (Centre Informatique National de l'Enseignement Supérieur).

## 4.2. Enhanced visual reconstruction by Gaussian Splatting

While photogrammetry requires many images to accurately reconstruct a scene and produce detailed and accurate 3D models, there are several alternative techniques like Neural Radiance Fields (NeRF) (Mildenhall et al., 2021) or 3D gaussian splatting (Kerbl et al., 2023) that can be an effective solution when quick content production is needed or when realistic 360° imaging is required for example. In the context of cultural heritage, several NeRF methods have been used on different cultural heritage datasets: smartphone videos, touristic approaches, or reflex cameras. Among them, Instant-NGP (Muller et al., 2022) and Nerfacto (Tancik et al., 2023) methods achieved the best outcomes (Mazzaca et al., 2023). Several studies have showed that NeRFs could be used for rendering objects (sculptures, archaeological remains, sites, paintings etc.) that are challenging for photogrammetry (Condorelli et al., 2021; Croce et al., 2023). NeRF has been successfully used for the documentation of cultural heritage sites. A study comparing NeRF and Multi-View Stereo Structure From Motion (MVS-SfM) found that while NeRF requires fewer images to produce accurate models, MVS-SfM provides more precise reconstructions of the structure of the sites (Balloni et al., 2023). As far as our knowledge, GSPLAT has never been used on cultural heritage dataset. Therefore, in this work, we decided to used GSPLAT on our data.

3D Gaussian splatting is a volume rendering technique that has emerged as a promising alternative to NeRFs for real-time rendering of 3D scenes. Unlike NeRFs, which rely on neural networks for scene representation, 3D Gaussian splatting utilizes a sparse set of 3D Gaussian functions, also known as "splats," to approximate the scene's radiance field. This approach offers several advantages, including improved rendering speed, reduced memory usage, and the ability for direct editing of the scene geometry. 3DGS represent complex 3D surfaces using a collection of discrete splats. Each splat is defined by its center position, scale, and orientation, along with additional parameters that capture the scene's radiance information. By distributing these splats throughout the scene, the overall radiance field can be approximated with a high degree of accuracy. The rendering process in 3D Gaussian splatting involves evaluating the radiance contribution from each splat that is visible from the current viewing point. To determine the visibility of a splat, the technique utilizes a visibility-aware splatting algorithm that efficiently assesses the occlusion relationships between splats. Once the visible splats are identified, their radiance contributions are combined to generate the final image at the current viewing point (Figure 6).

<figure><img src="image_7.png" /><figcaption>Figure 6. 3D ellipsoid for GSPLAT (left) ; GSPLAT (right)</figcaption></figure>

These techniques offer different advantages and trade-offs compared to photogrammetry. 3D Gaussian Splatting achieve high-quality rendering with faster training and real-time

performance, particularly for complex scenes and high-resolution outputs. However, 3D Gaussian Splatting may struggle with scenes that have many occlusions or areas with little texture.

Our results on Notre-Dame objects show that there are still some limitations. We were able to observe transparency problems on some of our objects. In addition, this method was not designed to process large quantities of high-definition images from photogrammetry, making calculation times particularly long.

## 5. CONCLUSION

In this paper we presented a full acquisition-processing-publication workflow that can be adapted to different object typologies and volumes, and takes into account the traceability of information. From acquisition to dissemination, we have put in place solutions to overcome most of the obstacles inherent in a corpus such as that of Notre Dame remains. So far, we have been able to confirm its effectiveness on several different groups of artifacts : 80 timbers and 20 metallic staples acquired over 5 campaigns spread over 2023, but also 120 stone fragments acquired between 2021 and 2022. More than 1000 items are planned to be digitized en 2024. In addition, we are experimenting with its reinforcement through calibrated and strategic interventions of AI-based methods to streamline the workflow.

As we aim to make all digital data from the Notre Dame de Paris project available in an open-access format post-2025, we fully acknowledge the responsibility inherent in creating such a comprehensive digital corpus. Recognizing that no solution is perfect and digitizations are influenced by various priorities, transparency becomes a key aspect of our approach. It's essential to articulate our data collection process as clearly as possible, facilitating its broad dissemination.

Our ambition to harmonize and manage data is intrinsically tied to this sense of responsibility. That's why we offer complementary models to minimize the need for users to transform them for use. Each modification in the data should be accompanied by a corresponding update in the data information, maintaining integrity and clarity. Additionally, it's our duty to clearly delineate the limitations of our methods, such as by making processing errors accessible and explaining our choices transparently.

Another aspect of our responsibility is data efficiency. Given that our corpus will ultimately encompass several terabytes of data, it's crucial to avoid unnecessary redundancy. This means not only ensuring that saved data doesn't duplicate, but also defining and targeting the optimal use of the data we produce. We facilitate this by allowing users to engage with every stage of our workflow - from calibration and processing to exports and archiving. By upholding these standards, we not only fulfill our responsibility towards the digital heritage community but also contribute to the creation of a rich, accessible, and valuable digital resource that can be utilized and built upon well into the future.

## ACKNOWLEDGEMENTS

The Digital Data Working Group of the Notre Dame Scientific Action is supported by several institutions and projects. These include the Centre National de la Recherche Scientifique (CNRS), the French Ministry of Culture, the Établissement public chargé de la conservation et de la restauration de Notre-Dame de Paris (EPRNDP), the Fondation des Sciences du
digital archiving service provided by the CINES (Centre Informatique National de l'Enseignement Supérieur).

## 4.2. Enhanced visual reconstruction by Gaussian Splatting

While photogrammetry requires many images to accurately reconstruct a scene and produce detailed and accurate 3D models, there are several alternative techniques like Neural Radiance Fields (NeRF) (Mildenhall et al., 2021) or 3D gaussian splatting (Kerbl et al., 2023) that can be an effective solution when quick content production is needed or when realistic 360° imaging is required for example. In the context of cultural heritage, several NeRF methods have been used on different cultural heritage datasets: smartphone videos, touristic approaches, or reflex cameras. Among them, Instant-NGP (Muller et al., 2022) and Nerfacto (Tancik et al., 2023) methods achieved the best outcomes (Mazzaca et al., 2023). Several studies have showed that NeRFs could be used for rendering objects (sculptures, archaeological remains, sites, paintings etc.) that are challenging for photogrammetry (Condorelli et al., 2021; Croce et al., 2023). NeRF has been successfully used for the documentation of cultural heritage sites. A study comparing NeRF and Multi-View Stereo Structure From Motion (MVS-SfM) found that while NeRF requires fewer images to produce accurate models, MVS-SfM provides more precise reconstructions of the structure of the sites (Balloni et al., 2023). As far as our knowledge, GSPLAT has never been used on cultural heritage dataset. Therefore, in this work, we decided to used GSPLAT on our data.

3D Gaussian splatting is a volume rendering technique that has emerged as a promising alternative to NeRFs for real-time rendering of 3D scenes. Unlike NeRFs, which rely on neural networks for scene representation, 3D Gaussian splatting utilizes a sparse set of 3D Gaussian functions, also known as "splats," to approximate the scene's radiance field. This approach offers several advantages, including improved rendering speed, reduced memory usage, and the ability for direct editing of the scene geometry. 3DGS represent complex 3D surfaces using a collection of discrete splats. Each splat is defined by its center position, scale, and orientation, along with additional parameters that capture the scene's radiance information. By distributing these splats throughout the scene, the overall radiance field can be approximated with a high degree of accuracy. The rendering process in 3D Gaussian splatting involves evaluating the radiance contribution from each splat that is visible from the current viewing point. To determine the visibility of a splat, the technique utilizes a visibility-aware splatting algorithm that efficiently assesses the occlusion relationships between splats. Once the visible splats are identified, their radiance contributions are combined to generate the final image at the current viewing point (Figure 6).

<figure><img src="image_7.png" /><figcaption>Figure 6. 3D ellipsoid for GSPLAT (left) ; GSPLAT (right)</figcaption></figure>

These techniques offer different advantages and trade-offs compared to photogrammetry. 3D Gaussian Splatting achieve high-quality rendering with faster training and real-time

performance, particularly for complex scenes and high-resolution outputs. However, 3D Gaussian Splatting may struggle with scenes that have many occlusions or areas with little texture.

Our results on Notre-Dame objects show that there are still some limitations. We were able to observe transparency problems on some of our objects. In addition, this method was not designed to process large quantities of high-definition images from photogrammetry, making calculation times particularly long.

## 5. CONCLUSION

In this paper we presented a full acquisition-processing-publication workflow that can be adapted to different object typologies and volumes, and takes into account the traceability of information. From acquisition to dissemination, we have put in place solutions to overcome most of the obstacles inherent in a corpus such as that of Notre Dame remains. So far, we have been able to confirm its effectiveness on several different groups of artifacts : 80 timbers and 20 metallic staples acquired over 5 campaigns spread over 2023, but also 120 stone fragments acquired between 2021 and 2022. More than 1000 items are planned to be digitized en 2024. In addition, we are experimenting with its reinforcement through calibrated and strategic interventions of AI-based methods to streamline the workflow.

As we aim to make all digital data from the Notre Dame de Paris project available in an open-access format post-2025, we fully acknowledge the responsibility inherent in creating such a comprehensive digital corpus. Recognizing that no solution is perfect and digitizations are influenced by various priorities, transparency becomes a key aspect of our approach. It's essential to articulate our data collection process as clearly as possible, facilitating its broad dissemination.

Our ambition to harmonize and manage data is intrinsically tied to this sense of responsibility. That's why we offer complementary models to minimize the need for users to transform them for use. Each modification in the data should be accompanied by a corresponding update in the data information, maintaining integrity and clarity. Additionally, it's our duty to clearly delineate the limitations of our methods, such as by making processing errors accessible and explaining our choices transparently.

Another aspect of our responsibility is data efficiency. Given that our corpus will ultimately encompass several terabytes of data, it's crucial to avoid unnecessary redundancy. This means not only ensuring that saved data doesn't duplicate, but also defining and targeting the optimal use of the data we produce. We facilitate this by allowing users to engage with every stage of our workflow - from calibration and processing to exports and archiving. By upholding these standards, we not only fulfill our responsibility towards the digital heritage community but also contribute to the creation of a rich, accessible, and valuable digital resource that can be utilized and built upon well into the future.

## ACKNOWLEDGEMENTS

The Digital Data Working Group of the Notre Dame Scientific Action is supported by several institutions and projects. These include the Centre National de la Recherche Scientifique (CNRS), the French Ministry of Culture, the Établissement public chargé de la conservation et de la restauration de Notre-Dame de Paris (EPRNDP), the Fondation des Sciences du
Patrimoine (FSP) and the European Research Council (ERC)
Advanced Grant nDame_Heritage project funded by the
European Union.

REFERENCES

Abergel, V., Manuel, A., Pamart, A., Cao, I., De Luca, L., 2023. Aïoli: A reality-based 3D annotation cloud platform for the collaborative documentation of cultural heritage artefacts. *Digital Applications in Archaeology and Cultural Heritage*, 30, e00285.

Alliez, P., Bergerot, L., Bernard, J.-F., Boust, C., Bruseker, G., Carboni, N., Chayani, M., et al., 2016. Digital 3D Objects in Art and Humanities: Challenges of Creation, Interoperability and Preservation, White Paper: A Result of the PARTHENOS Workshop Held in Bordeaux at Maison Des Sciences de l'Homme d'Aquitaine and at Archeovision Lab.

Ball, P., 2020. The huge scientific effort to study Notre-Dame's ashes. *Nature*, 577(7789), 153–155.

Balloni, E., Gorgoglione, L., Paolanti, M., Mancini, A., & Pierdicca, R., 2023. Few shot photogrammetry: a comparison between NERF and MVS-SFM for the documentation of cultural heritage. *The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences*, 48, 155–162.

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., ... & Liang, P., 2021. On the opportunities and risks of foundation models. *arXiv preprint* arXiv:2108.07258.

Condorelli, F., Rinaudo, F., Salvadore, F., Tagliaventi, S., 2021. A comparison between 3d reconstruction using NERF neural networks and MVS algorithms on cultural heritage images. *The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences*. XLIII-B2-2021. 565-570. 10.5194/isprs-archives-XLIII-B2-2021-565-2021.

Croce, V., Caroti, G., De Luca, L., Piemonte, A., & Véron, P., 2023. Neural radiance field (NERF): review and potential applications to digital cultural heritage. *The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences*, 48, 453–460.

De Luca, L., 2024. A digital ecosystem for the multidisciplinary study of Notre-Dame de Paris. Journal of Cultural Heritage. Special Issue - Notre-Dame de Paris : a multidisciplinary scientifique site. *Journal of Cultural Heritage, Special Issue Notre-Dame de Paris: a multidisciplinary scientific site*. Elsevier.

De Luca, L., Abergel, V., Guillem, A., Malavergne, O., Manuel, A., Neroulidis, A., Roussel, R., Roussel, M., Tournon, S., 2022. L'écosystème numérique n-dame pour l'analyse et la mémorisation multi-dimensionnelle du chantier scientifique Notre-Dame-de-Paris. SCAN'22 - 10e Séminaire de Conception Architecturale Numérique.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., ... & Houlsby, N., 2020. An image is worth 16x16 words: Transformers for image recognition at scale. *arXiv preprint* arXiv:2010.11929.

Dutailly, B., Tournon, S., Chayani, M., Grimaud, V., Granier, X., 2023. aLTAG3D: A User-Friendly Metadata Documentation Software. *DARIAH Annual Event 2023: Cultural Heritage Data as Humanities Research Data?*.

Gaiani, M., Apollonio, F.I., Ballabeni, A., Remondino, F., 2017. Securing Color Fidelity in 3D Architectural Heritage Scenarios. *Sensors* 2017, 17, 2437.

Gaiani, M., Ballabeni, A., 2018. SHAFT (SAT & HUE Adaptive Fine Tuning), a new automated solution for target-based color correction. *Colour and Colorimetry Multidisciplinary Contributions*, XIVB, 69–80.

Guillem, A, Gros, A, and De Luca, L., 2023. Faire Parler Les Claveaux Effondrés de La Cathédrale Notre-Dame de Paris : Un Récit Fondé Sur Le Web Sémantique et Les Linked Open Data.

He, K., Chen, X., Xie, S., Li, Y., Dollár, P., & Girshick, R., 2022. Masked autoencoders are scalable vision learners. *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 16000–16009.

Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G., 2023. 3d gaussian splatting for real-time radiance field rendering. *ACM Transactions on Graphics (ToG)*, 42(4), 1–14.

Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y. et al., 2023. Segment anything. *arXiv preprint* arXiv:2304.02643.

Liu, J., Ram, S., 2017. Improving the Domain Independence of Data Provenance Ontologies: A Demonstration Using Conceptual Graphs and the W7 Model. J. Database Manage. 28, 1 (January 2017), 43–62.

Mazzacca, G., Karami, A., Rigon, S., Farella, E. M., Trybala, P., & Remondino, F., 2023. NERF for heritage 3D reconstruction. *The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences*, 48, 1051–1058.

Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R., 2021. Nerf: Representing scenes as neural radiance fields for view synthesis. *Communications of the ACM*, 65(1), 99–106.

Müller, T., Evans, A., Schied, C., & Keller, A., 2022. Instant neural graphics primitives with a multiresolution hash encoding. *ACM Transactions on Graphics (ToG)*, 41(4), 1–15.

Néroulidis, A., Pouyet, T., Tournon, S., Roussel, M., Callieri, M., Manuel, A., Abergel, V., Malavergne, O., Cao, I., Roussel, R., Granier, X., Rodier, X., De Luca, L., 2024. A digital platform for the centralization and long-term preservation of multidisciplinary scientific data belonging to the Notre Dame de Paris scientific action. *Journal of Cultural Heritage, Special Issue Notre-Dame de Paris: a multidisciplinary scientific site*. Elsevier.

Pamart, A., Morlet, F., De Luca, L., Véron, P., 2020. A Robust and Versatile Pipeline for Automatic Photogrammetric-Based Registration of Multimodal Cultural Heritage Documentation. *Remote Sensing*, 12, 2051.

Pamart, A., De Luca, L., Véron, P., 2022. A metadata enriched system for the documentation of multi-modal digital imaging surveys. *Studies in Digital Heritage* 6, 1–24.
Patrimoine (FSP) and the European Research Council (ERC)
Advanced Grant nDame_Heritage project funded by the
European Union.

REFERENCES

Abergel, V., Manuel, A., Pamart, A., Cao, I., De Luca, L., 2023.
Äioli: A reality-based 3D annotation cloud platform for the
collaborative documentation of cultural heritage artefacts.
*Digital Applications in Archaeology and Cultural Heritage*, 30,
e00285.

Alliez, P., Bergerot, L., Bernard, J.-F., Boust, C., Bruseker, G., Carboni, N., Chayani, M., et al., 2016. Digital 3D Objects in Art and Humanities: Challenges of Creation, Interoperability and Preservation, White Paper: A Result of the PARTHENOS Workshop Held in Bordeaux at Maison Des Sciences de l'Homme d'Aquitaine and at Archeovision Lab.

Ball, P., 2020. The huge scientific effort to study Notre-Dame's
ashes. *Nature*, 577(7789), 153–155.

Balloni, E., Gorgoglione, L., Paolanti, M., Mancini, A., &
Pierdicca, R., 2023. Few shot photogrammetry: a comparison
between NERF and MVS-SFM for the documentation of
cultural heritage. *The International Archives of the
Photogrammetry, Remote Sensing and Spatial Information
Sciences*, 48, 155–162.

Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., ... & Liang, P., 2021. On the opportunities and risks of foundation models. *arXiv preprint* arXiv:2108.07258.

Condorelli, F., Rinaudo, F., Salvadore, F., Tagliaventi, S., 2021.
A comparison between 3d reconstruction using NERF neural
networks and MVS algorithms on cultural heritage images. *The
International Archives of the Photogrammetry, Remote Sensing
and Spatial Information Sciences*. XLIII-B2-2021. 565-570.
10.5194/isprs-archives-XLIII-B2-2021-565-2021.

Croce, V., Caroti, G., De Luca, L., Piemonte, A., & Véron, P., 2023. Neural radiance field (NERF): review and potential applications to digital cultural heritage. *The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences*, 48, 453–460.

De Luca, L., 2024. A digital ecosystem for the multidisciplinary
study of Notre-Dame de Paris. Journal of Cultural Heritage.
Special Issue - Notre-Dame de Paris : a multidisciplinary
scientifique site. *Journal of Cultural Heritage, Special Issue
Notre-Dame de Paris: a multidisciplinary scientific site*.
Elsevier.

De Luca, L., Abergel, V., Guillem, A., Malavergne, O., Manuel, A., Neroulidis, A., Roussel, R., Roussel, M., Tournon, S., 2022. L'écosystème numérique n-dame pour l'analyse et la mémorisation multi-dimensionnelle du chantier scientifique Notre-Dame-de-Paris. SCAN'22 - 10e Séminaire de Conception Architecturale Numérique.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., ... & Houlsby, N., 2020. An image is worth 16x16 words: Transformers for image recognition at scale. *arXiv preprint* arXiv:2010.11929.

Dutailly, B., Tournon, S., Chayani, M., Grimaud, V., Granier, X., 2023. aLTAG3D: A User-Friendly Metadata Documentation Software. *DARIAH Annual Event 2023: Cultural Heritage Data as Humanities Research Data* ?.

Gaiani, M., Apollonio, F.I., Ballabeni, A., Remondino, F., 2017. Securing Color Fidelity in 3D Architectural Heritage Scenarios. *Sensors* 2017, 17, 2437.

Gaiani, M., Ballabeni, A., 2018. SHAFT (SAT & HUE Adaptive Fine Tuning), a new automated solution for target-based color correction. *Colour and Colorimetry Multidisciplinary Contributions*, XIVB, 69–80.

Guillem, A, Gros, A, and De Luca, L., 2023. Faire Parler Les Claveaux Effondrés de La Cathédrale Notre-Dame de Paris : Un Récit Fondé Sur Le Web Sémantique et Les Linked Open Data.

He, K., Chen, X., Xie, S., Li, Y., Dollár, P., & Girshick, R., 2022. Masked autoencoders are scalable vision learners. *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition*, 16000–16009.

Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G., 2023. 3d gaussian splatting for real-time radiance field rendering. *ACM Transactions on Graphics (ToG)*, 42(4), 1–14.

Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y. et al., 2023. Segment anything. *arXiv preprint* arXiv:2304.02643.

Liu, J., Ram, S., 2017. Improving the Domain Independence of Data Provenance Ontologies: A Demonstration Using Conceptual Graphs and the W7 Model. J. Database Manage. 28, 1 (January 2017), 43–62.

Mazzacca, G., Karami, A., Rigon, S., Farella, E. M., Trybala, P., & Remondino, F., 2023. NERF for heritage 3D reconstruction. *The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences*, 48, 1051–1058.

Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R., 2021. Nerf: Representing scenes as neural radiance fields for view synthesis. *Communications of the ACM*, 65(1), 99–106.

Müller, T., Evans, A., Schied, C., & Keller, A., 2022. Instant neural graphics primitives with a multiresolution hash encoding. *ACM Transactions on Graphics (ToG)*, 41(4), 1–15.

Néroulidis, A., Pouyet, T., Tournon, S., Roussel, M., Callieri, M., Manuel, A., Abergel, V., Malavergne, O., Cao, I., Roussel, R., Granier, X., Rodier, X., De Luca, L., 2024. A digital platform for the centralization and long-term preservation of multidisciplinary scientific data belonging to the Notre Dame de Paris scientific action. *Journal of Cultural Heritage, Special Issue Notre-Dame de Paris: a multidisciplinary scientific site*. Elsevier.

Pamart, A., Morlet, F., De Luca, L., Véron, P., 2020. A Robust and Versatile Pipeline for Automatic Photogrammetric-Based Registration of Multimodal Cultural Heritage Documentation. *Remote Sensing*, 12, 2051.

Pamart, A., De Luca, L., Véron, P., 2022. A metadata enriched system for the documentation of multi-modal digital imaging surveys. *Studies in Digital Heritage* 6, 1–24.