# Hybrid Spine Simulator Prototype for X-ray Free Pedicle Screws Fixation Training
## 2 Materials and Methods
### Simulator Testing
#### 2.3.1 Evaluation of the Camera Calibration and AR Visualization  
Three fiducial spherical landmarks (2 mm in diameter) were added to each verteb model at known positions to evaluate the cameras' calibration: one was positioned at the vertebral body so that it is visible by the lateral camera (Cam\({}_{\text{Lat}}\)), and the other two at the vertebral peduncle and the spinous process so that are visible by the top camera, Cam\({}_{\text{Top}}\) (Figure 8). The VR models of landmarks were added to the AR scene for estimating the target visualization error (TVE), thus evaluating the accuracy of the AR overlay (a similar proce  
Figure 8: Testing setup: (**a**, **left**) configuration of the two cameras and the spine model; (**b**, **right**) a close-up of the spine model with 3D-printed spherical landmarks, and the trocar mockup. Figure 7: Screenshots of the User Interface: (**a**, **left**) the User Interface during a calibration procedure; (**b**, **right**) the User Interface during a simulation session. This second screenshot shows both the virtual fluoroscopic image (from the selected Virtual C-Arm pose) and the AR view from the top camera (Cam\({}_{\text{Top}}\)). dure was adopted in [16] to evaluate the accuracy of a hybrid simulator). With this aim, the vertebral models were moved in ten different positions, acquiring each time the AR image of both virtual cameras. More specifically, for each position, two images were captured from each virtual camera. The first image was acquired with AR switched-off; therefore, this image contains only the scene as viewed by the camera, without AR information. The second image was acquired with AR switched-on, thus it also shows the virtual landmarks. This produced for each camera two sets of corresponding images: we refer to the set with AR switched-off as the \"Real Set\", and to the set with AR switched-on as the \"AR Set\". The correspondence between the Real Set and the AR Set guarantees an accurate modeling of the camera projection models and their intrinsic and extrinsic calibration, which is a requirement for the realism and fidelity of both the AR Visualization and the Virtual X-ray Visualization modalities. The acquired sets were automatically processed to estimate the 2D target visualization error (IVE2D), that is: the offset, expressed in pixels, between the virtual and real objects in the image plane (i.e., the centroids of the virtual landmarks, and the corresponding real 3D-printed landmarks). The Real Set and the AR Set were processed in the hue-saturation-value (HSV) color space that allows a robust segmentation even when the target objects show non-uniformities in illumination, shadows, or shading [37]. The image processing, performed in MATLAB, included the following steps:  
1. Transformation from RGB to HSV color space. 2. Detection of the virtual landmarks in the AR Set (Figure 9a). 3. Detection of the 3D-printed landmarks in the Real Set (Figure 9b). Virtual landmarks were detected with the circular Hough transform, using their diameter as an input parameter. The Hough transform sensitivity was adjusted to detect all the landmarks in the image (four markers for Cam\({}_{\text{Lat}}\) and eight for Cam\({}_{\text{Top}}\)): an increase in the sensitivity allows the detection of more circular objects, including weak and partially obscured circles (however higher sensitivity values also increase the risk of false detection) [38]. The knowledge of the virtual markers position derived from step 2 was used to improve the robustness of step 3. Each 3D-printed landmark indeed was searched within a region of interest (ROI) centered on the position of the corresponding virtual landmark. Figure 9: Examples of corresponding images of AR Set and Real Set from the Cam\({}_{\text{Top}}\) view. (**a**, **left**) Image from the AR Set: the eight detected digital markers are circled in red, and a zoomed view of the first top left marker is provided.