# SIFT-Based Depth Estimation for Accurate 3D Reconstruction in Cultural Heritage Preservation
## 2 Materials and Methods
### Experimental Setup  
This research aims to study tangible cultural heritage, e.g., sculptures in high relief and round relief. The contribution of this research can be divided into three major parts: (1) collecting and recording 2D images of tangible cultural heritage using a low-cost digital camera (DSLR); (2) constructing 3D models of the tangible cultural heritage using the input dataset; and (3) creating a collection of 3D models of tangible cultural heritage in digital format or in 3D printed format. The expected benefits of this work are two-fold: (1) the 3D archiving of tangible cultural heritage and (2) improving the accuracy of 3D models as reference data for the restoration of tangible cultural heritage in the future. Figure 4 illustrates the proposed method. Figure 3: 3D Model with Texture [37]. All images were taken from The National Art Museum of China, Beijing. The image dataset included sculptures, models, and artifacts. Input images were used to compute point clouds based on SIFT feature extraction [37]. The properties of SIFT include all possible coordinates and colors of the point cloud. The redundant and unnecessary point clouds were removed later. Therefore, the mesh model was generated, and the color map of the point cloud was constructed in this step. To construct the point cloud, monocular depth estimation was also used (monocular depth estimation is a computer vision technique used to predict the depth information of the scene) for depth map computation. Global-local path networks for monocular depth estimation [43] were used in the proposed method to generate monocular colors for the depth map. These colors were then used to define the depth property of the point cloud. In the final step, the Open3D library was used to generate the point cloud based on the color and depth information extracted from the input images. To construct the smooth surface of the mesh, the Poisson surface reconstruction algorithm [44] was also applied in the mesh generation process. Poisson distribution was applied to the point cloud construction. Figure 5 shows the global-local path networks for monocular depth estimation proposed by [43]. The block diagram of the proposed method also addresses the practical image capture challenges for optimal 3D reconstruction to ensure the successful reconstruction of tangible cultural heritage; careful image capture techniques are essential for stereo image matching. The incorporation of a calibration target into the scene was also considered to guarantee consistent camera viewpoints, facilitating automatic feature correspondence during image matching. Furthermore, a capture strategy was adopted, promoting an adequate image overlap. Some options were used, such as utilizing a fixed camera rig for consistent positioning or systematically capturing a sequence of overlapping images with the careful monitoring of camera locations. Experiments were conducted using various camera angles to strike a balance between sufficient overlap and the capture of intricate details from diverse perspectives. Moreover, image capture software was employed for real-time overlap feedback to optimize the image capture process, preventing both deficient and excessive overlap. Moreover, dealing with environmental conditions, particularly lighting, significantly impacted the success of the reconstruction process. Whenever possible, images in the museum were captured where controlled lighting could be maintained to reduce shadows and uneven illumination, ensuring consistent feature identification. If consistent lighting was unattainable, the capture of multiple image sets was considered under various lighting conditions. Therefore, the proposed approach requires configuration and a setting up process to ensure empowerment of the subsequent selection of the images yielding the most favorable reconstruction outcomes compared to SfM, MVS, and software-dependent methods like photogrammetry (AgiSoft, Reality Capture (Unreal), and ReCap). Figure 4: Block diagram of the proposed method.