## 3 Case Study: Interactive Modelling Implemented Using Vario XR-4 and Ultraleap Motion Controller
### Implementation Details
Here are some of the C# commands that enable access to the data of the right and left recognised hands, using the Leap namespaces:  
* to access the hand information from an acquired frame:  
Left hand \(\rightarrow\)**Hand** leftHand = frame. **GetHand(Chirality.Left)**;  
Right hand \(\rightarrow\)**Hand** rightHand = frame. **GetHand(Chirality.Right)**;  
* to access the coordinates of the \(j\)-th finger of the right hand (0 = thumb, 1 = index, 2 = middle 3 = ring, 4 = pinky):  
**Vector3** tipCoordinate = rightHand.**fingers[j]. **TipPosition**;  
* to access the coordinates of the joint of the \(k\)-th bone of the \(j\)-th finger of the right hand (0 = metacarpal, 1 = proximal, 2 = intermediate, 3 = distal)  
**Vector3** jointCoordinates = rightHand.**fingers[j].**bones[k].**NextJoint. **ToVector3()**;  
* to access the angle between two adjacent bones (bone1 and bone2) on \(j\)-th finger of the right hand:  
**Bone** bone1 = rightHand.**fingers[j]. **bones[k1]**;  
**Bone** bone2 = rightHand.**fingers[j]. **bones[k2]**;  
**float** angle = **Vector3.Angle(bone1.Direction.ToVector3(),bone2.Direction.ToVector3())**;  
* to access the coordinates of the middle of the palm or the right hand:  
**Vector3** palmPosition = rightHand. **PalmPosition**;  
* to access the normal vector of the palm of the right hand:  
**Vector3** palmAxis = rightHand. **PalmAxis**;  
* to access the coordinates of the pinch point of the right hand:  
**Vector3** pinchPoint = rightHand. **GetPinchPosition()**;  
The recognition of the hand pose is achieved by comparing the angle between adjacent bones with those of a reference pose, within a specific tolerance. For the case study, we used the _Pose Detector_ object provided in the Ultraleap's library for the direct comparison with standard poses (pinch, fist, thumb-up, thumb-down). The initial control cage is generated as a shape of a rectangular box of 0.5 \(\times\) 0.5 \(\times\) 0.5 m dimensions. The parameters \(n_{x}\), \(n_{y}\), \(n_{z}\) specifying the number of cage subdivisions along the three main axes are chosen as 5, 5, 5. The number of vertices \(N_{Vertex}\) defining the initial control cage is calculated as the difference between the total vertices \(N_{Vertex}^{Tot}\) generated by the nested loops and the hidden vertices \(N_{Vertex}^{NS}\), which are internal to the cage itself:  
\[N_{Vertex}=N_{Vertex}^{Tot}-N_{Vertex}^{NS}=(n_{x}+1)\big{(}n_{y}+1\big{)}(n_ {z}+1)-(n_{x}-1)\big{(}n_{y}-1\big{)}(n_{z}-1) \tag{11}\]  
The number of edges \(N_{Edge}\) and the number of faces \(N_{Face}\) of the control cage are expressed by Equation (12) and Equation (13), respectively. \[N_{Edge}=2n_{x}\big{(}n_{y}-1\big{)}+2n_{x}(n_{z}+1)+2n_{y}(n_{z}-1)+2n_{y}(n_ {x}+1)+2n_{z}(n_{x}-1)+2n_{z}(n_{y}+1) \tag{12}\]  
\[N_{Face}=2n_{x}n_{y}+2n_{y}n_{z}+2n_{x}n_{z} \tag{13}\]  
The vertices and the centroids of the faces are represented by spheres, while the edges are identified by cylinders. The assignment of the \(F_{Num}\) and \(F_{ID}\) to each face is managed using the parameters \(n_{x}\), \(n_{y}\), \(n_{z}\), and the parameters \(\delta_{x}\), \(\delta_{y}\), \(\delta_{z}\), which indicate the variation in the indices of adjacent vertices along the \(x\), \(y\), and \(z\) directions, respectively. Figure 7 shows an initial photo (a) and 6 frames captured by the Varjo XR-4 cameras during the AR experience of the case study. In the first step (b), the user explores the virtual  
scene and locates the starting geometry collimating it to the real environment. In the first frame (b), it is possible to see how depth occlusion with the user's hands is active, which returns great realism. The control cage is depicted in blue, the control vertices in red, the edges of the cage in blue, and the subdivided surface in red. Points and lines are depicted as three-dimensional geometries in order to give a spatial awareness. Green spheres indicate the centres of the control cage faces to facilitate the selection and grabbing.