# Augmented Reality: Mapping Methods and Tools for Enhancing the Human Role in Healthcare HMI
## 3 Results
### Intraoperative Solution
#### 3.3.1 Lower and Upper Limbs
[66]. This system uses an optical camera and a computer to project an image of the exposed bone in the surgical field onto the image obtained from the preoperative CT scan during surgery. In contrast to this system, the work of [PERSON] et al. [71] requires no preoperative images. Using a smartphone and 2D barcode markers, their AR-KNEE system allows overlay of the tibial and anteroposterior long axis on the surgical field. In this way, the center of the femoral head and the femoral mechanical axis are seen directly on the surgical field, allowing the surgeon to recognize an incorrect registration by comparing the femoral mechanical axis to the tibial line. The AR-KNEE system was demonstrated to achieve an error \(<1^{\circ}\) in the resection of the proximal part of the tibia in both the coronal and the sagittal planes, as well as the accuracy in the resection of the distal part of the femur, evaluated by the same team in an experimental setting [40], was also found to be \(<1^{\circ}\) in both coronal and sagittal planes. In addition, they compared a distal femoral resection performed with conventional intramedullary guidance and the AR-KNEE system in a clinical setting and found that the AR-KNEE system ensured considerably higher accuracy. A different system was developed by [PERSON] et al. [30], using AR as a guide for TKA. The footprint of the system comprises smart glasses, two small disposable sensors, and a control unit that managed tracking information collected by a pocket-sized wireless optical tracking system. A new feature enabled the intraoperative measurement of the alignment and positioning effects of the prosthesis on soft tissue balance, in order to assess whether the best implant position and limb axis are achieved. [PERSON] et al. [26] proposed an in-situ AR navigation tool based on a reconstruction of the 3D model of the knee using preoperative imaging. The preoperative 3D model of the knee is updated with tissue properties considering the actually adopted intraoperative positions. Then, the obtained model is shown through a glasses-free 3D display for navigation, a stereo display that, thanks to stereopsis, is capable of transmitting depth perception to the viewer. In this way, an accurate 3D structure of the tissue was obtained to allow surgical AR navigation, giving the surgeon a more detailed view of the patient's 3D anatomy. [PERSON] et al. [82] used an AR-based surgical navigation (ARSN) system with hybrid electromagnetic and optical tracking that provided 3D images using an integral videography approach for intramedullary nail locking. They were able to correctly set eighteen drills in five tibia models and one leg phantom during their pre-clinical study. A similar system was provided by [PERSON] et al. [41], who proposed a hybrid HoloLens-to-world methodology for the registration exploiting a custom-made registration cube and an external EM tracker. The feasibility of this approach was proved in a cadaver experiment using an ARSN system based on Microsoft HoloLens 2 as a support for surgeons to complete distal interlocking, indicating that the system satisfies the accuracy requirement of distal interlocking while reducing the total time for intramedullary nailing. [PERSON] et al. [32] developed an HMD-based AR navigation platform to assist surgeons in orthopedic procedures involving the drilling of exposed bone structures. The application was deployed on HoloLens 1, includes markerless tracking of target anatomy, and supports visualization in both optical see-through (OST) and video see-through (VST) paradigms. The navigation performance of the system was evaluated by a user study both quantitatively and qualitatively, obtaining that AR visualizations can be a reliable tool for manual guidance of femoral drilling. Regarding the upper limb, [PERSON] et al. [45] reported the successful integration of AR into elbow arthroscopy. Their system uses AR technology to superimpose nerve data obtained from preoperative images, which are essential for successful elbow surgery, on an arthroscopy monitor.