## 3 Case Study: Interactive Modelling Implemented Using Vario XR-4 and Ultraleap Motion Controller
### Implementation Details  
The headset chosen for the implementation of the case study is the Vario XR-4. It is a high-end mixed-reality helmet equipped with two mini-LED displays with a resolution of 3840 \(\times\) 3744 at 51 PPD and a 90 Hz refresh rate. It is equipped with 20 MP pass-through cameras. The spatial awareness is achieved by a 300 KPix LiDAR with 7 m range. It does not have its own hand-tracking system, but it can be achieved by using 2 nd generation Ultraleap's Leap Motion Controller to be integrated with a proprietary mounting kit. This device has two infrared cameras and multiple infrared LEDs. For each hand, the controller is able to track 24 joints, returning the spatial coordinates of 4 finger joints for each of the 5 fingers (knuckle, proximal, intermediate, and distal) and 5 additional joints for the palm and wrist. The 2 nd generation device has improved performance with respect to its 1 st generation that was extensively used and assessed by the scientific community [38,41]. The manufacturer provides development libraries that allow direct access to the tracking information that becomes necessary to be able to apply the algorithm and procedures described in the previous sections. The great advantage in using the Vario XR-4 is not only in the high technical specifications but above all in the availability of support libraries (Vario XR Plugin and Varjo SDK) made available by the manufacturer that allow extensive and complete access. The libraries have several extremely useful features for customising mixed-reality environments and allow almost complete control of the potential offered by the instrumentation. More specifically, the Vario XR Plugin gives access to the _VarjoMixedReality_ class to manage the Vario interface and to the _VarjoRendering_ class to manage rendering options. The AR functionality of the Vario XR-4 can be enabled with the following two scripted commands:  
**VarjoMixedReality.StartRender()**;  
**VarjoRendering.SetOpaque(false)**;  
The first line starts the rendering of the scene on the two displays and the second line enables the pass-through (rendering of the video frame acquired by the cameras). To ensure high visual realism, it is advisable to take into account depth occlusion [39] which allows the correct stacking between real objects and virtual objects, preventing the overlapping of virtual objects if they are at a greater distance from real objects than the observer's point of view. This verification can be managed directly from the libraries supplied with the Varjo by activating the calculation of the depth map calling the respective methods in the _VarjoMixedReality_ class by using the script function following:  
**VarjoMixedReality.EnableDepthEstimation()**;  
The calculation of depth occlusions is then performed using data from the lidar sensor. Although the depth occlusion is computationally demanding, it has a great contribution to the realism of the scene. However, note that how you enable mixed reality can vary from headset to headset, and not all headsets directly support depth map calculation, which is a quite time-consuming activity. As for hand tracking, Ultraleap Motion Controller has a library that allows full access to both raw data (position and attitude of hand bones and joint) or elaborated data (pose and gesture estimation). In the case of the example, we extracted and processed raw data that are more suitable for implementing the adapted GAF/OAF methodology. The communication with the controller is managed by the _Leap Service Provider_ that is ableto retrieve synchronous information that are stored in a _frame_ data structure. For each acquisition frame, the _frame_ object is updated and then the information can be extracted by the corresponding methods and properties. For example, it is possible to extract the vector of the recognised hands which contains 5 fingers which contain information on the connecting structures (bones and joints).