# Realistic Texture Mapping of 3D Medical Models Using RGBD Camera for Mixed Reality Applications
## 4 Discussions
The second study proposed an approach based on a monocular camera, 3D-2D feature mapping, and surface parameterization technology for texture surface reconstruction, reporting an accuracy of 0.32 mm. Despite the interesting results from these studies, employing a rigid mounting system between the RGBD system and CBCT machine significantly limits its general applicability. Replicating the position and methodology across all CBCT machines is not guaranteed. A methodology like the one described in this study could help address this limitation: relying on manual movement of the RGBD system ensures a low registration error while maintaining flexibility. The second part of the study focused on the visualization and interaction of the coloured 3D mesh, created with MeshLab software, in an augmented reality environment through the Microsoft HoloLens 2 head-mount display. The development of a software application for visualization and interaction on the Microsoft HoloLens 2 device, showcases the practical application of the proposed method. Natural hand gestures for manipulation and scaling, contribute to a more immersive and user-friendly experience. Since the described pointcloud processing and registration method does not rely on specific tools or binding techniques, it could be used and replicated with medical images acquired from other types of medical exams like magnetic resonance or standard computed tomography and with different external texture acquisition techniques like photogrammetry or structured light cameras. Visualization of a realistic textured 3D model reconstructed from medical images could provide doctors and surgeons with highly clinically relevant information, especially in areas where visual concept and patient aesthetics maintenance are crucial, such as maxillical surgery or dentistry. Another area of application is during image-guided surgical planning stages, where accurate and careful planning of skin entry points and trajectories is crucial for a correct outcome. Additionally, the increasing introduction of technologies like augmented, mixed, and virtual reality in the medical field holds promise for the future of these technologies. Their potential is undeniable and could allow physicians and surgeons to perform image analysis or surgical planning phases innovatively and user-friendly through hand-gesture integration. Despite the shown results, this study has some limitations. The use of the ChArUco marker for determining the camera's position, placed behind the human-head mimicking phantom, limits the range of movement with the RGBD camera. Specifically, to ensure precise registration of the pointcloud, the marker needs to be within the camera's field of view, which can make capturing images of the nuchal region difficult. To overcome this challenge, incorporating two or more strategically positioned ChArUco markers within the scene could be a solution. Despite most of the points in the RGBD pointcloud being correctly registered (Figure 14), there are still areas where registration occurs incorrectly, especially in the nose and ear regions. Choosing a phantom with less uniform external texture and ensuring careful illumination during the RGBD pointcloud acquisition phases could help overcome this issue. The pointcloud alignment technique described in this study was tested on the phantom shown in Figure 1. Replicating this methodology on multiple phantoms with different surface characteristics could be useful for highlighting the potential and limitations of this technique. Finally, the phantom (Figure 1) was hollow inside, thus lacking any structure to simulate internal tissues. The applicability of the method proposed in this study remains valid, as it is based on the segmentation of the outer contour of the CBCT-scanned object, but future studies could test it on phantoms specifically designed to be scanned with radiographic techniques. This further step forward could also enable interaction with such internal structures, even in the AR visualization phase.