# Realistic Texture Mapping of 3D Medical Models Using RGBD Camera for Mixed Reality Applications
## 1 Introduction  
Augmented reality (AR) combines elements of the real world with digital elements, overlaying virtual information onto the view of the surrounding environment through devices such as smartphones, glasses, or headsets. Mixed reality (MR) is an advanced form of AR that integrates digital elements more deeply with the physical environment, allowing virtual objects to interact in real-time with the real world and vice versa through, for example, user hand gestures. Virtual reality (VR) creates a completely immersive environment separate from the physical reality. Users wear VR headsets to be fully immersed in virtual worlds, isolated from the external environment. The use of these technologies in the industry is spreading, with applications being developed not only for entertainment and video games but also in many other areas, including medicine and surgery. The surgical areas where these technologies are being introduced are different: orthopaedic surgery [1; 2; 3; 4; 5; 6], vascular surgery [7; 8; 9], oncology [10; 11; 12; 13; 14], neurosurgery [15; 16; 17; 18] and maxillofacial surgery [19; 20; 21; 22]. AR, MR and VR can also be used for educational purposes. These technologies are being developed to improve and facilitate the learning of complex information, such as learning in physiology and anatomy, where students need a three-dimensional knowledge of human organ systems and structures [23; 24; 25] but also to train, test and improve medical students' practical skills and competencies through surgical simulations [26; 27; 28; 29; 30]. During surgical operations, AR/MR could provide real-time information to surgeons directly in the operating room, making the use of external screens for imaging potentially useless, which would be all incorporated into the viewer. Moreover, they can enhance the doctor's view by adding useful information for tools orientation in the surgical site, overlapping virtual models on the patient, whenever the area of interest is visible or not, and also giving information about the surgical planning and its course during the operation. The reconstruction of a 3D digital model from images obtained with cone-beam computed tomography (CBCT) machines and displayed using these technologies, could provide doctors with an innovative and intuitive visualization method, informing them about the health status of internal tissues. Moreover, it could offer a new way to carry out the activity of image-guided surgical planning, which is crucial to achieving desired treatment outcomes [31]. Other areas of interest are dentistry and maxillofacial surgery where the untextured soft tissue makes it difficult for both the surgeon and the patient to construct a visual concept of the surgical plan [32], additionally, preserving the patient's aesthetic appearance could be of great interest. However, since CT technology allows the acquisition of internal tissue images, the reconstructed 3D model wouldn't have an external texture faithful to the real one. The integration of this information into the digital model could be useful for creating a new digital model that is more realistic and similar to the real one. Adding external texture could assist doctors during training phases by providing them with digital models that closely resemble those they will interact with during real surgical procedures. Attempting to overcome this limitation, especially for craniofacial research, several studies have been conducted trying to acquire, through different technologies, the outer surface of the subject [33]. In particular, consumer-grade 3D scanning, such as RGB-D (red-green-blue-depth) sensors, which combine red, green, and blue (RGB) data with depth information (D) have been tested. The Azure Kinect development kit (Azure Kinect DK, Microsoft Inc., Redmond, WA, USA), released in 2020, is based on a continuous-wave time of flight (ToF) camera and seems to be a viable 3D scanning solution for clinical and research applications, with a systematic error of less than 2 mm [34; 35].